[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:02:38.385541  8021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.213.126:34287
I20260709 13:02:38.387662  8021 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:02:38.388892  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:38.408311  8028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:38.408452  8030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:38.408895  8027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:38.410333  8021 server_base.cc:1061] running on GCE node
I20260709 13:02:39.637367  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:39.637625  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:39.637778  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602159637761 us; error 0 us; skew 500 ppm
I20260709 13:02:39.638542  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:39.644205  8021 webserver.cc:533] Webserver started at http://127.7.213.126:45753/ using document root <none> and password file <none>
I20260709 13:02:39.645046  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:39.645246  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:39.645651  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:39.651415  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/master-0-root/instance:
uuid: "1951c9338083445db62cf3b5bfa3ff07"
format_stamp: "Formatted at 2026-07-09 13:02:39 on dist-test-slave-9g29"
I20260709 13:02:39.659404  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:02:39.664904  8036 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:39.666292  8021 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:02:39.666798  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/master-0-root
uuid: "1951c9338083445db62cf3b5bfa3ff07"
format_stamp: "Formatted at 2026-07-09 13:02:39 on dist-test-slave-9g29"
I20260709 13:02:39.667158  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:39.724355  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:39.726107  8021 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:02:39.726639  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:39.805521  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.126:34287
I20260709 13:02:39.805598  8087 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.126:34287 every 8 connection(s)
I20260709 13:02:39.813875  8088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:39.837819  8088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07: Bootstrap starting.
I20260709 13:02:39.844282  8088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:39.846302  8088 log.cc:826] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07: Log is configured to *not* fsync() on all Append() calls
I20260709 13:02:39.851074  8088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07: No bootstrap required, opened a new log
I20260709 13:02:39.871635  8088 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1951c9338083445db62cf3b5bfa3ff07" member_type: VOTER }
I20260709 13:02:39.872231  8088 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:39.872529  8088 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1951c9338083445db62cf3b5bfa3ff07, State: Initialized, Role: FOLLOWER
I20260709 13:02:39.873472  8088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1951c9338083445db62cf3b5bfa3ff07" member_type: VOTER }
I20260709 13:02:39.874007  8088 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:02:39.874315  8088 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:02:39.874677  8088 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:39.879155  8088 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1951c9338083445db62cf3b5bfa3ff07" member_type: VOTER }
I20260709 13:02:39.880024  8088 leader_election.cc:304] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1951c9338083445db62cf3b5bfa3ff07; no voters: 
I20260709 13:02:39.881811  8088 leader_election.cc:290] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:02:39.882217  8091 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:39.885087  8091 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 1 LEADER]: Becoming Leader. State: Replica: 1951c9338083445db62cf3b5bfa3ff07, State: Running, Role: LEADER
I20260709 13:02:39.885983  8091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1951c9338083445db62cf3b5bfa3ff07" member_type: VOTER }
I20260709 13:02:39.886404  8088 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:02:39.897775  8092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1951c9338083445db62cf3b5bfa3ff07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1951c9338083445db62cf3b5bfa3ff07" member_type: VOTER } }
I20260709 13:02:39.898540  8092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:39.898581  8093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1951c9338083445db62cf3b5bfa3ff07. Latest consensus state: current_term: 1 leader_uuid: "1951c9338083445db62cf3b5bfa3ff07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1951c9338083445db62cf3b5bfa3ff07" member_type: VOTER } }
I20260709 13:02:39.899204  8093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:39.901597  8101 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:02:39.913275  8101 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:02:39.915988  8021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:02:39.931468  8101 catalog_manager.cc:1383] Generated new cluster ID: aecf535fac9a405283a34b65a7f2c3a4
I20260709 13:02:39.931826  8101 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:02:39.949432  8101 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:02:39.951084  8101 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:02:39.967662  8101 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07: Generated new TSK 0
I20260709 13:02:39.968878  8101 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:02:39.982618  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:39.992200  8110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:39.993708  8111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:39.996919  8113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:39.998214  8021 server_base.cc:1061] running on GCE node
I20260709 13:02:39.999217  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:39.999536  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:39.999785  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602159999760 us; error 0 us; skew 500 ppm
I20260709 13:02:40.000684  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:40.005264  8021 webserver.cc:533] Webserver started at http://127.7.213.65:38437/ using document root <none> and password file <none>
I20260709 13:02:40.006096  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:40.006376  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:40.006779  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:40.008476  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-0-root/instance:
uuid: "932d80a534fa4e7db50b464603e2b9e3"
format_stamp: "Formatted at 2026-07-09 13:02:39 on dist-test-slave-9g29"
I20260709 13:02:40.016374  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:02:40.022135  8118 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.023097  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.001s
I20260709 13:02:40.023494  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "932d80a534fa4e7db50b464603e2b9e3"
format_stamp: "Formatted at 2026-07-09 13:02:39 on dist-test-slave-9g29"
I20260709 13:02:40.024003  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:40.047597  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:40.049589  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:40.052714  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:40.058863  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:40.059226  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.059592  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:40.059870  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.002s	sys 0.000s
I20260709 13:02:40.156819  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:38991
I20260709 13:02:40.156972  8180 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:38991 every 8 connection(s)
I20260709 13:02:40.164872  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:40.182011  8185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:40.194736  8186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:40.197098  8188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:40.196861  8021 server_base.cc:1061] running on GCE node
I20260709 13:02:40.198392  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:40.198683  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:40.198889  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602160198870 us; error 0 us; skew 500 ppm
I20260709 13:02:40.199692  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:40.202996  8021 webserver.cc:533] Webserver started at http://127.7.213.66:38001/ using document root <none> and password file <none>
I20260709 13:02:40.203714  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:40.204005  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:40.204365  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:40.205313  8181 heartbeater.cc:344] Connected to a master server at 127.7.213.126:34287
I20260709 13:02:40.205828  8181 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:40.205925  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-1-root/instance:
uuid: "9d7cd04e5edc4ade87478888855a363e"
format_stamp: "Formatted at 2026-07-09 13:02:40 on dist-test-slave-9g29"
I20260709 13:02:40.207039  8181 heartbeater.cc:507] Master 127.7.213.126:34287 requested a full tablet report, sending...
I20260709 13:02:40.210847  8053 ts_manager.cc:194] Registered new tserver with Master: 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991)
I20260709 13:02:40.213294  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.008s	sys 0.001s
I20260709 13:02:40.213867  8053 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55670
I20260709 13:02:40.218644  8193 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.219955  8021 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:02:40.220255  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-1-root
uuid: "9d7cd04e5edc4ade87478888855a363e"
format_stamp: "Formatted at 2026-07-09 13:02:40 on dist-test-slave-9g29"
I20260709 13:02:40.220556  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:40.247114  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:40.248523  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:40.251214  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:40.254226  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:40.254415  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.254722  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:40.254868  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.353430  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.66:35645
I20260709 13:02:40.353541  8255 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.66:35645 every 8 connection(s)
I20260709 13:02:40.360103  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:40.368479  8259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:40.370069  8260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:40.377085  8256 heartbeater.cc:344] Connected to a master server at 127.7.213.126:34287
I20260709 13:02:40.377544  8256 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:40.378427  8021 server_base.cc:1061] running on GCE node
I20260709 13:02:40.378626  8256 heartbeater.cc:507] Master 127.7.213.126:34287 requested a full tablet report, sending...
W20260709 13:02:40.379156  8263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:40.380242  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:40.380501  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:40.380699  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602160380681 us; error 0 us; skew 500 ppm
I20260709 13:02:40.381042  8053 ts_manager.cc:194] Registered new tserver with Master: 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:40.381376  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:40.382804  8053 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55672
I20260709 13:02:40.385212  8021 webserver.cc:533] Webserver started at http://127.7.213.67:39547/ using document root <none> and password file <none>
I20260709 13:02:40.385725  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:40.385924  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:40.386195  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:40.387297  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-2-root/instance:
uuid: "788c1c01145a4d8e951fafe5cad72cd7"
format_stamp: "Formatted at 2026-07-09 13:02:40 on dist-test-slave-9g29"
I20260709 13:02:40.392159  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:02:40.395678  8267 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.396860  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:02:40.397166  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-2-root
uuid: "788c1c01145a4d8e951fafe5cad72cd7"
format_stamp: "Formatted at 2026-07-09 13:02:40 on dist-test-slave-9g29"
I20260709 13:02:40.397478  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602158212224-8021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:40.415270  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:40.416707  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:40.418205  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:40.421204  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:40.421440  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.421733  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:40.421924  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:40.522033  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.67:33329
I20260709 13:02:40.522159  8329 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.67:33329 every 8 connection(s)
I20260709 13:02:40.547806  8330 heartbeater.cc:344] Connected to a master server at 127.7.213.126:34287
I20260709 13:02:40.548324  8330 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:40.549439  8330 heartbeater.cc:507] Master 127.7.213.126:34287 requested a full tablet report, sending...
I20260709 13:02:40.552237  8053 ts_manager.cc:194] Registered new tserver with Master: 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:40.553287  8021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.025041678s
I20260709 13:02:40.553880  8053 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55684
I20260709 13:02:40.611564  8053 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55688:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:02:40.615525  8053 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:02:40.775481  8295 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee3fb267edc49bf838bac8d18d65d14 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:40.777262  8295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee3fb267edc49bf838bac8d18d65d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.777571  8219 tablet_service.cc:1511] Processing CreateTablet for tablet c6fb6e3565064bd1a87124ed6b20c3d5 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:40.779132  8219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6fb6e3565064bd1a87124ed6b20c3d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.778857  8221 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee3fb267edc49bf838bac8d18d65d14 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:40.780234  8221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee3fb267edc49bf838bac8d18d65d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.779912  8215 tablet_service.cc:1511] Processing CreateTablet for tablet 93ac48387e5a4acabd3aedc7f2a6270a (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:40.781391  8215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ac48387e5a4acabd3aedc7f2a6270a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.781752  8218 tablet_service.cc:1511] Processing CreateTablet for tablet 56143748b832452c83027f7a0016323f (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:40.782305  8220 tablet_service.cc:1511] Processing CreateTablet for tablet aa4c1171ee57481880d302faecbfb085 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:40.783193  8218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56143748b832452c83027f7a0016323f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.783869  8220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa4c1171ee57481880d302faecbfb085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.784482  8217 tablet_service.cc:1511] Processing CreateTablet for tablet 53ddf1f9a83248d1bb76ce958ca6fce3 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:40.785247  8145 tablet_service.cc:1511] Processing CreateTablet for tablet aa4c1171ee57481880d302faecbfb085 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:40.785827  8217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53ddf1f9a83248d1bb76ce958ca6fce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.786830  8145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa4c1171ee57481880d302faecbfb085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.791801  8143 tablet_service.cc:1511] Processing CreateTablet for tablet 56143748b832452c83027f7a0016323f (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:40.789273  8146 tablet_service.cc:1511] Processing CreateTablet for tablet 3ee3fb267edc49bf838bac8d18d65d14 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:02:40.800438  8140 tablet_service.cc:1511] Processing CreateTablet for tablet 93ac48387e5a4acabd3aedc7f2a6270a (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:40.803447  8143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56143748b832452c83027f7a0016323f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.798771  8216 tablet_service.cc:1511] Processing CreateTablet for tablet 013e2fecd579459c92f501140d58c5a5 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:40.801905  8294 tablet_service.cc:1511] Processing CreateTablet for tablet aa4c1171ee57481880d302faecbfb085 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:02:40.819257  8294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa4c1171ee57481880d302faecbfb085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.797960  8141 tablet_service.cc:1511] Processing CreateTablet for tablet 013e2fecd579459c92f501140d58c5a5 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:40.803982  8142 tablet_service.cc:1511] Processing CreateTablet for tablet 53ddf1f9a83248d1bb76ce958ca6fce3 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:40.822963  8146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ee3fb267edc49bf838bac8d18d65d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.824487  8289 tablet_service.cc:1511] Processing CreateTablet for tablet 93ac48387e5a4acabd3aedc7f2a6270a (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:02:40.803807  8144 tablet_service.cc:1511] Processing CreateTablet for tablet c6fb6e3565064bd1a87124ed6b20c3d5 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:40.833081  8216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013e2fecd579459c92f501140d58c5a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.830150  8293 tablet_service.cc:1511] Processing CreateTablet for tablet c6fb6e3565064bd1a87124ed6b20c3d5 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:02:40.834372  8292 tablet_service.cc:1511] Processing CreateTablet for tablet 56143748b832452c83027f7a0016323f (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:02:40.833071  8140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ac48387e5a4acabd3aedc7f2a6270a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.836421  8293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6fb6e3565064bd1a87124ed6b20c3d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.831238  8291 tablet_service.cc:1511] Processing CreateTablet for tablet 53ddf1f9a83248d1bb76ce958ca6fce3 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:02:40.839954  8141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013e2fecd579459c92f501140d58c5a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.834383  8290 tablet_service.cc:1511] Processing CreateTablet for tablet 013e2fecd579459c92f501140d58c5a5 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:02:40.845153  8289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93ac48387e5a4acabd3aedc7f2a6270a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.850127  8142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53ddf1f9a83248d1bb76ce958ca6fce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.851665  8290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013e2fecd579459c92f501140d58c5a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.852370  8144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6fb6e3565064bd1a87124ed6b20c3d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.874382  8139 tablet_service.cc:1511] Processing CreateTablet for tablet 903a6b1533f64dd9afe641c52fc62c9e (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:40.878212  8139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903a6b1533f64dd9afe641c52fc62c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.884528  8291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53ddf1f9a83248d1bb76ce958ca6fce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.908190  8292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56143748b832452c83027f7a0016323f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:02:40.909974  8100 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:40.922472  8214 tablet_service.cc:1511] Processing CreateTablet for tablet 903a6b1533f64dd9afe641c52fc62c9e (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:40.925845  8214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903a6b1533f64dd9afe641c52fc62c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.956220  8138 tablet_service.cc:1511] Processing CreateTablet for tablet 1e61de98e1e1405c8c9bade40e860281 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:40.959374  8138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e61de98e1e1405c8c9bade40e860281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.945818  8288 tablet_service.cc:1511] Processing CreateTablet for tablet 903a6b1533f64dd9afe641c52fc62c9e (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:02:40.964674  8288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 903a6b1533f64dd9afe641c52fc62c9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:40.987500  8213 tablet_service.cc:1511] Processing CreateTablet for tablet 1e61de98e1e1405c8c9bade40e860281 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:40.993937  8213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e61de98e1e1405c8c9bade40e860281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:41.045009  8348 tablet_bootstrap.cc:492] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e: Bootstrap starting.
I20260709 13:02:41.044952  8287 tablet_service.cc:1511] Processing CreateTablet for tablet 1e61de98e1e1405c8c9bade40e860281 (DEFAULT_TABLE table=test-workload [id=d5a69b8e5c244adf95271fdbe7f41e56]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:02:41.050105  8287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e61de98e1e1405c8c9bade40e860281. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:41.053148  8349 tablet_bootstrap.cc:492] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7: Bootstrap starting.
I20260709 13:02:41.061214  8349 tablet_bootstrap.cc:654] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.061751  8350 tablet_bootstrap.cc:492] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3: Bootstrap starting.
I20260709 13:02:41.099108  8348 tablet_bootstrap.cc:654] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.126842  8349 tablet_bootstrap.cc:492] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7: No bootstrap required, opened a new log
I20260709 13:02:41.127303  8349 ts_tablet_manager.cc:1403] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent bootstrapping tablet: real 0.075s	user 0.003s	sys 0.021s
I20260709 13:02:41.133003  8348 tablet_bootstrap.cc:492] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e: No bootstrap required, opened a new log
I20260709 13:02:41.133716  8348 ts_tablet_manager.cc:1403] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.005s
I20260709 13:02:41.133891  8350 tablet_bootstrap.cc:654] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.135425  8349 raft_consensus.cc:359] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.136888  8349 raft_consensus.cc:385] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.137204  8349 raft_consensus.cc:740] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.137768  8348 raft_consensus.cc:359] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.140740  8348 raft_consensus.cc:385] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.141106  8348 raft_consensus.cc:740] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.142107  8349 consensus_queue.cc:260] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.142102  8348 consensus_queue.cc:260] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.148223  8350 tablet_bootstrap.cc:492] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3: No bootstrap required, opened a new log
I20260709 13:02:41.148938  8350 ts_tablet_manager.cc:1403] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3: Time spent bootstrapping tablet: real 0.088s	user 0.008s	sys 0.009s
I20260709 13:02:41.152180  8348 ts_tablet_manager.cc:1434] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.012s
I20260709 13:02:41.152484  8256 heartbeater.cc:499] Master 127.7.213.126:34287 was elected leader, sending a full tablet report...
I20260709 13:02:41.153337  8348 tablet_bootstrap.cc:492] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e: Bootstrap starting.
I20260709 13:02:41.160128  8348 tablet_bootstrap.cc:654] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.161701  8330 heartbeater.cc:499] Master 127.7.213.126:34287 was elected leader, sending a full tablet report...
I20260709 13:02:41.152166  8350 raft_consensus.cc:359] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.162813  8350 raft_consensus.cc:385] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.163110  8350 raft_consensus.cc:740] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.163862  8350 consensus_queue.cc:260] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.162077  8349 ts_tablet_manager.cc:1434] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.019s
I20260709 13:02:41.166806  8349 tablet_bootstrap.cc:492] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7: Bootstrap starting.
I20260709 13:02:41.173691  8349 tablet_bootstrap.cc:654] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.190577  8181 heartbeater.cc:499] Master 127.7.213.126:34287 was elected leader, sending a full tablet report...
I20260709 13:02:41.190886  8350 ts_tablet_manager.cc:1434] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3: Time spent starting tablet: real 0.041s	user 0.012s	sys 0.005s
I20260709 13:02:41.192010  8350 tablet_bootstrap.cc:492] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3: Bootstrap starting.
I20260709 13:02:41.203365  8348 tablet_bootstrap.cc:492] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e: No bootstrap required, opened a new log
I20260709 13:02:41.203670  8350 tablet_bootstrap.cc:654] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.204072  8348 ts_tablet_manager.cc:1403] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e: Time spent bootstrapping tablet: real 0.051s	user 0.025s	sys 0.018s
I20260709 13:02:41.207125  8348 raft_consensus.cc:359] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.207964  8348 raft_consensus.cc:385] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.208305  8348 raft_consensus.cc:740] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.209172  8348 consensus_queue.cc:260] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.211313  8348 ts_tablet_manager.cc:1434] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:02:41.212311  8348 tablet_bootstrap.cc:492] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e: Bootstrap starting.
I20260709 13:02:41.217996  8349 tablet_bootstrap.cc:492] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7: No bootstrap required, opened a new log
I20260709 13:02:41.218475  8349 ts_tablet_manager.cc:1403] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.001s
I20260709 13:02:41.219133  8348 tablet_bootstrap.cc:654] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.221469  8349 raft_consensus.cc:359] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.222354  8349 raft_consensus.cc:385] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.222661  8349 raft_consensus.cc:740] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.223404  8349 consensus_queue.cc:260] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.225533  8349 ts_tablet_manager.cc:1434] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:02:41.226382  8349 tablet_bootstrap.cc:492] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7: Bootstrap starting.
I20260709 13:02:41.232023  8348 tablet_bootstrap.cc:492] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e: No bootstrap required, opened a new log
I20260709 13:02:41.232551  8348 ts_tablet_manager.cc:1403] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.010s
I20260709 13:02:41.235595  8348 raft_consensus.cc:359] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.236502  8350 tablet_bootstrap.cc:492] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3: No bootstrap required, opened a new log
I20260709 13:02:41.236496  8348 raft_consensus.cc:385] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.237129  8350 ts_tablet_manager.cc:1403] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.005s
I20260709 13:02:41.237138  8348 raft_consensus.cc:740] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.238180  8348 consensus_queue.cc:260] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.240710  8348 ts_tablet_manager.cc:1434] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 13:02:41.241715  8348 tablet_bootstrap.cc:492] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e: Bootstrap starting.
I20260709 13:02:41.242338  8349 tablet_bootstrap.cc:654] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.247239  8350 raft_consensus.cc:359] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:41.248106  8350 raft_consensus.cc:385] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.248406  8350 raft_consensus.cc:740] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.248947  8348 tablet_bootstrap.cc:654] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.249128  8350 consensus_queue.cc:260] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:41.251498  8350 ts_tablet_manager.cc:1434] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:02:41.252420  8350 tablet_bootstrap.cc:492] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3: Bootstrap starting.
I20260709 13:02:41.258862  8350 tablet_bootstrap.cc:654] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.263023  8349 tablet_bootstrap.cc:492] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7: No bootstrap required, opened a new log
I20260709 13:02:41.263521  8349 ts_tablet_manager.cc:1403] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.000s
I20260709 13:02:41.266333  8349 raft_consensus.cc:359] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.268317  8349 raft_consensus.cc:385] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.268631  8349 raft_consensus.cc:740] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.269344  8349 consensus_queue.cc:260] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.269806  8348 tablet_bootstrap.cc:492] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e: No bootstrap required, opened a new log
I20260709 13:02:41.270407  8348 ts_tablet_manager.cc:1403] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.003s
I20260709 13:02:41.273465  8348 raft_consensus.cc:359] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.271680  8349 ts_tablet_manager.cc:1434] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:02:41.274253  8348 raft_consensus.cc:385] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.274616  8348 raft_consensus.cc:740] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.275144  8349 tablet_bootstrap.cc:492] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7: Bootstrap starting.
I20260709 13:02:41.275362  8348 consensus_queue.cc:260] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.277626  8348 ts_tablet_manager.cc:1434] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:02:41.278512  8348 tablet_bootstrap.cc:492] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e: Bootstrap starting.
I20260709 13:02:41.284801  8348 tablet_bootstrap.cc:654] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.285981  8349 tablet_bootstrap.cc:654] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:41.291463  8331 tablet.cc:2406] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:02:41.299103  8349 tablet_bootstrap.cc:492] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7: No bootstrap required, opened a new log
I20260709 13:02:41.299602  8349 ts_tablet_manager.cc:1403] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.004s
I20260709 13:02:41.299787  8350 tablet_bootstrap.cc:492] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3: No bootstrap required, opened a new log
I20260709 13:02:41.299546  8356 raft_consensus.cc:493] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.300251  8348 tablet_bootstrap.cc:492] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e: No bootstrap required, opened a new log
I20260709 13:02:41.300714  8348 ts_tablet_manager.cc:1403] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.012s
I20260709 13:02:41.300387  8356 raft_consensus.cc:515] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:41.302655  8349 raft_consensus.cc:359] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.303359  8349 raft_consensus.cc:385] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.303642  8349 raft_consensus.cc:740] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.303637  8348 raft_consensus.cc:359] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.304641  8348 raft_consensus.cc:385] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.304972  8348 raft_consensus.cc:740] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.304541  8349 consensus_queue.cc:260] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.305769  8348 consensus_queue.cc:260] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.307586  8349 ts_tablet_manager.cc:1434] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 13:02:41.308647  8349 tablet_bootstrap.cc:492] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7: Bootstrap starting.
I20260709 13:02:41.308593  8356 leader_election.cc:290] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:41.321868  8354 raft_consensus.cc:493] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.322376  8354 raft_consensus.cc:515] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.309545  8355 raft_consensus.cc:493] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.328534  8349 tablet_bootstrap.cc:654] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.327462  8355 raft_consensus.cc:515] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.331287  8354 leader_election.cc:290] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:41.336364  8350 ts_tablet_manager.cc:1403] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.009s
I20260709 13:02:41.341682  8350 raft_consensus.cc:359] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.352545  8350 raft_consensus.cc:385] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.354137  8350 raft_consensus.cc:740] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.316973  8348 ts_tablet_manager.cc:1434] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:02:41.357281  8348 tablet_bootstrap.cc:492] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e: Bootstrap starting.
I20260709 13:02:41.350910  8355 leader_election.cc:290] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:41.360008  8355 raft_consensus.cc:493] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.360550  8355 raft_consensus.cc:515] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.362545  8355 leader_election.cc:290] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:41.371471  8350 consensus_queue.cc:260] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.384718  8348 tablet_bootstrap.cc:654] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.418783  8350 ts_tablet_manager.cc:1434] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3: Time spent starting tablet: real 0.081s	user 0.009s	sys 0.000s
I20260709 13:02:41.424417  8350 tablet_bootstrap.cc:492] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3: Bootstrap starting.
I20260709 13:02:41.432461  8348 tablet_bootstrap.cc:492] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e: No bootstrap required, opened a new log
I20260709 13:02:41.432969  8348 ts_tablet_manager.cc:1403] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e: Time spent bootstrapping tablet: real 0.076s	user 0.015s	sys 0.011s
I20260709 13:02:41.433861  8349 tablet_bootstrap.cc:492] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7: No bootstrap required, opened a new log
I20260709 13:02:41.434795  8349 ts_tablet_manager.cc:1403] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7: Time spent bootstrapping tablet: real 0.126s	user 0.045s	sys 0.060s
I20260709 13:02:41.444999  8350 tablet_bootstrap.cc:654] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.452133  8349 raft_consensus.cc:359] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.455432  8349 raft_consensus.cc:385] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.454864  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:41.456597  8349 raft_consensus.cc:740] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.459436  8348 raft_consensus.cc:359] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:41.460827  8348 raft_consensus.cc:385] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.461228  8348 raft_consensus.cc:740] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.462991  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:41.463018  8349 consensus_queue.cc:260] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
W20260709 13:02:41.474573  8121 leader_election.cc:343] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:41.477187  8119 leader_election.cc:343] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.476691  8349 ts_tablet_manager.cc:1434] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.001s
I20260709 13:02:41.465309  8348 consensus_queue.cc:260] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:41.477860  8119 leader_election.cc:304] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3; no voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:41.485653  8356 raft_consensus.cc:2749] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:41.493287  8349 tablet_bootstrap.cc:492] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7: Bootstrap starting.
I20260709 13:02:41.504664  8156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
W20260709 13:02:41.508606  8271 leader_election.cc:343] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.508733  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:41.509672  8348 ts_tablet_manager.cc:1434] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e: Time spent starting tablet: real 0.076s	user 0.006s	sys 0.005s
I20260709 13:02:41.509613  8231 raft_consensus.cc:2468] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 0.
I20260709 13:02:41.510926  8348 tablet_bootstrap.cc:492] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e: Bootstrap starting.
I20260709 13:02:41.513618  8354 raft_consensus.cc:493] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.514770  8354 raft_consensus.cc:515] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.517421  8268 leader_election.cc:304] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:41.520954  8354 leader_election.cc:290] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:41.526505  8156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:41.532804  8354 raft_consensus.cc:2804] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.533172  8354 raft_consensus.cc:493] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:41.533619  8271 leader_election.cc:343] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.534881  8374 raft_consensus.cc:493] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.533510  8354 raft_consensus.cc:3060] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.535389  8374 raft_consensus.cc:515] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.537106  8350 tablet_bootstrap.cc:492] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3: No bootstrap required, opened a new log
I20260709 13:02:41.540402  8350 ts_tablet_manager.cc:1403] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3: Time spent bootstrapping tablet: real 0.119s	user 0.018s	sys 0.015s
I20260709 13:02:41.541122  8348 tablet_bootstrap.cc:654] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.545826  8374 leader_election.cc:290] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:41.545847  8350 raft_consensus.cc:359] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.546638  8350 raft_consensus.cc:385] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.546886  8350 raft_consensus.cc:740] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.547467  8350 consensus_queue.cc:260] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.549095  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:41.549750  8231 raft_consensus.cc:2468] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 0.
I20260709 13:02:41.553121  8350 ts_tablet_manager.cc:1434] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:02:41.554860  8350 tablet_bootstrap.cc:492] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3: Bootstrap starting.
I20260709 13:02:41.554927  8268 leader_election.cc:304] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:41.553149  8354 raft_consensus.cc:515] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.558300  8382 raft_consensus.cc:2804] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.560766  8382 raft_consensus.cc:493] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.563865  8382 raft_consensus.cc:3060] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.564945  8349 tablet_bootstrap.cc:654] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.568706  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:41.570827  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:41.573201  8156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:41.573329  8304 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 0.
I20260709 13:02:41.571826  8354 leader_election.cc:290] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
W20260709 13:02:41.574710  8271 leader_election.cc:343] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.575284  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:41.576718  8348 tablet_bootstrap.cc:492] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e: No bootstrap required, opened a new log
I20260709 13:02:41.577291  8348 ts_tablet_manager.cc:1403] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e: Time spent bootstrapping tablet: real 0.067s	user 0.014s	sys 0.004s
I20260709 13:02:41.579226  8196 leader_election.cc:304] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:41.584162  8374 raft_consensus.cc:2804] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.580987  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:41.584654  8374 raft_consensus.cc:493] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.585075  8374 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:41.584087  8196 leader_election.cc:343] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.584846  8231 raft_consensus.cc:3060] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.592121  8350 tablet_bootstrap.cc:654] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.592027  8348 raft_consensus.cc:359] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.593086  8348 raft_consensus.cc:385] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.593737  8348 raft_consensus.cc:740] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.597952  8155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:41.598809  8156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:41.599434  8155 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 0.
I20260709 13:02:41.594726  8348 consensus_queue.cc:260] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
W20260709 13:02:41.600921  8197 leader_election.cc:343] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.597090  8382 raft_consensus.cc:515] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.602000  8374 raft_consensus.cc:515] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.602665  8348 ts_tablet_manager.cc:1434] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:02:41.603142  8305 raft_consensus.cc:2393] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9d7cd04e5edc4ade87478888855a363e in current term 1: Already voted for candidate 788c1c01145a4d8e951fafe5cad72cd7 in this term.
I20260709 13:02:41.599902  8356 raft_consensus.cc:493] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.604498  8356 raft_consensus.cc:515] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.605810  8348 tablet_bootstrap.cc:492] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e: Bootstrap starting.
I20260709 13:02:41.606884  8356 leader_election.cc:290] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:41.608779  8196 leader_election.cc:304] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d7cd04e5edc4ade87478888855a363e; no voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:41.610008  8355 raft_consensus.cc:3060] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.610760  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:41.608808  8230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:41.616523  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:41.613332  8374 leader_election.cc:290] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:41.617754  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:41.619268  8155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:41.617394  8231 raft_consensus.cc:2468] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 1.
W20260709 13:02:41.619799  8119 leader_election.cc:343] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:41.621920  8121 leader_election.cc:343] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.621871  8156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
W20260709 13:02:41.623502  8271 leader_election.cc:343] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.624068  8155 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.624869  8121 leader_election.cc:304] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3; no voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:41.625273  8382 leader_election.cc:290] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:41.625769  8356 raft_consensus.cc:2749] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:41.628361  8268 leader_election.cc:304] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:41.629614  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:41.621315  8304 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.631489  8382 raft_consensus.cc:2804] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:02:41.632650  8197 leader_election.cc:343] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.633703  8197 leader_election.cc:304] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d7cd04e5edc4ade87478888855a363e; no voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:41.633109  8382 raft_consensus.cc:697] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Becoming Leader. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:41.638302  8374 raft_consensus.cc:2749] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:41.643381  8348 tablet_bootstrap.cc:654] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.645308  8355 raft_consensus.cc:2749] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:41.645925  8304 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 1.
I20260709 13:02:41.647110  8196 leader_election.cc:304] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:41.647936  8374 raft_consensus.cc:2804] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.650827  8350 tablet_bootstrap.cc:492] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3: No bootstrap required, opened a new log
I20260709 13:02:41.651305  8350 ts_tablet_manager.cc:1403] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.009s
I20260709 13:02:41.637656  8382 consensus_queue.cc:237] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.652740  8155 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 1.
I20260709 13:02:41.658833  8349 tablet_bootstrap.cc:492] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7: No bootstrap required, opened a new log
I20260709 13:02:41.659369  8349 ts_tablet_manager.cc:1403] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent bootstrapping tablet: real 0.175s	user 0.024s	sys 0.012s
I20260709 13:02:41.665146  8231 raft_consensus.cc:2468] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 1.
I20260709 13:02:41.665350  8348 tablet_bootstrap.cc:492] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e: No bootstrap required, opened a new log
I20260709 13:02:41.666088  8348 ts_tablet_manager.cc:1403] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e: Time spent bootstrapping tablet: real 0.061s	user 0.008s	sys 0.013s
I20260709 13:02:41.666455  8355 raft_consensus.cc:493] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.667050  8355 raft_consensus.cc:515] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.669461  8349 raft_consensus.cc:359] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.672747  8355 leader_election.cc:290] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:41.673055  8349 raft_consensus.cc:385] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.672785  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:41.673491  8349 raft_consensus.cc:740] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.674046  8268 leader_election.cc:304] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:41.674230  8155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:41.675211  8354 raft_consensus.cc:2804] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.666582  8374 raft_consensus.cc:697] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:41.673719  8304 raft_consensus.cc:2468] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 0.
W20260709 13:02:41.676729  8197 leader_election.cc:343] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.684911  8196 leader_election.cc:304] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:41.674345  8349 consensus_queue.cc:260] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.669783  8350 raft_consensus.cc:359] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.686064  8348 raft_consensus.cc:359] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.686266  8355 raft_consensus.cc:2804] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.687402  8348 raft_consensus.cc:385] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.687659  8355 raft_consensus.cc:493] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.687728  8348 raft_consensus.cc:740] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.688220  8355 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.688616  8348 consensus_queue.cc:260] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.690717  8348 ts_tablet_manager.cc:1434] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.000s
I20260709 13:02:41.691614  8348 tablet_bootstrap.cc:492] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e: Bootstrap starting.
I20260709 13:02:41.679477  8382 raft_consensus.cc:493] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.686672  8350 raft_consensus.cc:385] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.693363  8350 raft_consensus.cc:740] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.693773  8387 raft_consensus.cc:493] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.694248  8350 consensus_queue.cc:260] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.694389  8387 raft_consensus.cc:515] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:41.692791  8382 raft_consensus.cc:515] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.709285  8387 leader_election.cc:290] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329), 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991)
I20260709 13:02:41.710391  8155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:41.710494  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:41.711087  8155 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 0.
I20260709 13:02:41.711102  8349 ts_tablet_manager.cc:1434] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.000s
I20260709 13:02:41.711217  8154 raft_consensus.cc:2468] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 0.
I20260709 13:02:41.724058  8271 leader_election.cc:304] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:41.715991  8354 raft_consensus.cc:697] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Becoming Leader. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:41.726739  8354 consensus_queue.cc:237] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.728125  8350 ts_tablet_manager.cc:1434] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3: Time spent starting tablet: real 0.076s	user 0.002s	sys 0.009s
I20260709 13:02:41.729889  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:41.730620  8350 tablet_bootstrap.cc:492] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3: Bootstrap starting.
I20260709 13:02:41.730574  8231 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 0.
I20260709 13:02:41.723453  8349 tablet_bootstrap.cc:492] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7: Bootstrap starting.
I20260709 13:02:41.735311  8197 leader_election.cc:304] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:41.738246  8387 raft_consensus.cc:2804] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.738683  8387 raft_consensus.cc:493] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.739053  8387 raft_consensus.cc:3060] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.711176  8374 consensus_queue.cc:237] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.725361  8382 leader_election.cc:290] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:41.747262  8389 raft_consensus.cc:2804] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.749862  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:41.748625  8052 catalog_manager.cc:5697] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:41.752491  8348 tablet_bootstrap.cc:654] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:41.756328  8196 leader_election.cc:343] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.759733  8391 raft_consensus.cc:493] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.754971  8389 raft_consensus.cc:493] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.773661  8389 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.760661  8391 raft_consensus.cc:515] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.780738  8387 raft_consensus.cc:515] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:41.784740  8387 leader_election.cc:290] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Requested vote from peers 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329), 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991)
I20260709 13:02:41.784806  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:41.785949  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
W20260709 13:02:41.786446  8196 leader_election.cc:343] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.786582  8154 raft_consensus.cc:3060] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.788142  8349 tablet_bootstrap.cc:654] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.801067  8154 raft_consensus.cc:2468] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 1.
I20260709 13:02:41.803134  8197 leader_election.cc:304] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 788c1c01145a4d8e951fafe5cad72cd7
I20260709 13:02:41.804630  8350 tablet_bootstrap.cc:654] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.809958  8374 raft_consensus.cc:2804] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.811842  8374 raft_consensus.cc:697] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:41.811540  8050 catalog_manager.cc:5697] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:41.813359  8374 consensus_queue.cc:237] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:41.802548  8355 raft_consensus.cc:515] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.817463  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:41.817727  8355 leader_election.cc:290] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
W20260709 13:02:41.819662  8196 leader_election.cc:343] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.820235  8155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:41.820961  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
W20260709 13:02:41.821764  8197 leader_election.cc:343] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.821781  8304 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.822278  8389 raft_consensus.cc:515] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.823241  8391 leader_election.cc:290] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:41.825788  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:41.827450  8154 raft_consensus.cc:2468] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 0.
I20260709 13:02:41.828912  8197 leader_election.cc:304] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 788c1c01145a4d8e951fafe5cad72cd7
I20260709 13:02:41.830049  8391 raft_consensus.cc:2804] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.830767  8391 raft_consensus.cc:493] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.831151  8391 raft_consensus.cc:3060] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.838451  8391 raft_consensus.cc:515] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.839181  8304 raft_consensus.cc:2468] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 1.
I20260709 13:02:41.841020  8196 leader_election.cc:304] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:41.842145  8355 raft_consensus.cc:2804] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.842677  8355 raft_consensus.cc:697] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:41.848851  8355 consensus_queue.cc:237] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.855428  8389 leader_election.cc:290] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:41.878096  8348 tablet_bootstrap.cc:492] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e: No bootstrap required, opened a new log
I20260709 13:02:41.878683  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:41.880093  8348 ts_tablet_manager.cc:1403] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e: Time spent bootstrapping tablet: real 0.189s	user 0.051s	sys 0.044s
I20260709 13:02:41.880622  8349 tablet_bootstrap.cc:492] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7: No bootstrap required, opened a new log
I20260709 13:02:41.852407  8155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:41.881269  8349 ts_tablet_manager.cc:1403] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent bootstrapping tablet: real 0.158s	user 0.013s	sys 0.004s
I20260709 13:02:41.881395  8155 raft_consensus.cc:3060] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.855662  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:41.888810  8349 raft_consensus.cc:359] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:41.890154  8349 raft_consensus.cc:385] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.890534  8349 raft_consensus.cc:740] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.883898  8154 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.891844  8231 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.898001  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:41.899955  8349 consensus_queue.cc:260] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
W20260709 13:02:41.901686  8196 leader_election.cc:343] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.902330  8349 ts_tablet_manager.cc:1434] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:02:41.913033  8349 tablet_bootstrap.cc:492] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7: Bootstrap starting.
I20260709 13:02:41.902426  8391 leader_election.cc:290] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:41.914845  8231 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 1.
I20260709 13:02:41.916154  8389 raft_consensus.cc:493] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.916461  8155 raft_consensus.cc:2468] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 1.
I20260709 13:02:41.916668  8389 raft_consensus.cc:515] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.916962  8268 leader_election.cc:304] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:41.917984  8197 leader_election.cc:304] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 788c1c01145a4d8e951fafe5cad72cd7
I20260709 13:02:41.918793  8355 raft_consensus.cc:2804] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.919046  8354 raft_consensus.cc:2804] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.918867  8348 raft_consensus.cc:359] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.919307  8355 raft_consensus.cc:697] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:41.919564  8348 raft_consensus.cc:385] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.919628  8354 raft_consensus.cc:697] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Becoming Leader. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:41.919915  8348 raft_consensus.cc:740] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.920207  8355 consensus_queue.cc:237] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.920617  8354 consensus_queue.cc:237] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
W20260709 13:02:41.913782  8100 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:41.918883  8389 leader_election.cc:290] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:41.926695  8155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e61de98e1e1405c8c9bade40e860281" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:41.928609  8348 consensus_queue.cc:260] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
W20260709 13:02:41.928262  8271 leader_election.cc:343] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:41.928169  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e61de98e1e1405c8c9bade40e860281" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:41.931514  8231 raft_consensus.cc:2468] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 0.
I20260709 13:02:41.932667  8154 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 1.
I20260709 13:02:41.932909  8268 leader_election.cc:304] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:41.935235  8348 ts_tablet_manager.cc:1434] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.001s
I20260709 13:02:41.935382  8389 raft_consensus.cc:2804] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:41.936046  8389 raft_consensus.cc:493] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.936327  8389 raft_consensus.cc:3060] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.949388  8389 raft_consensus.cc:515] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.950865  8350 tablet_bootstrap.cc:492] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3: No bootstrap required, opened a new log
I20260709 13:02:41.951944  8350 ts_tablet_manager.cc:1403] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3: Time spent bootstrapping tablet: real 0.222s	user 0.013s	sys 0.006s
I20260709 13:02:41.955147  8349 tablet_bootstrap.cc:654] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.956561  8389 leader_election.cc:290] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:41.960356  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e61de98e1e1405c8c9bade40e860281" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:41.961534  8231 raft_consensus.cc:3060] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.961158  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e61de98e1e1405c8c9bade40e860281" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:41.968604  8231 raft_consensus.cc:2468] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 1.
I20260709 13:02:41.971124  8268 leader_election.cc:304] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:41.971920  8389 raft_consensus.cc:2804] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:41.972355  8389 raft_consensus.cc:697] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Becoming Leader. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:41.973081  8389 consensus_queue.cc:237] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.961546  8350 raft_consensus.cc:359] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.974153  8350 raft_consensus.cc:385] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.974442  8350 raft_consensus.cc:740] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.975137  8350 consensus_queue.cc:260] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.977679  8350 ts_tablet_manager.cc:1434] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3: Time spent starting tablet: real 0.025s	user 0.000s	sys 0.008s
I20260709 13:02:41.978940  8350 tablet_bootstrap.cc:492] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3: Bootstrap starting.
I20260709 13:02:41.981876  8154 raft_consensus.cc:3060] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:41.982035  8349 tablet_bootstrap.cc:492] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7: No bootstrap required, opened a new log
I20260709 13:02:41.982615  8349 ts_tablet_manager.cc:1403] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7: Time spent bootstrapping tablet: real 0.070s	user 0.012s	sys 0.000s
I20260709 13:02:41.985872  8350 tablet_bootstrap.cc:654] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:41.985798  8349 raft_consensus.cc:359] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.986806  8349 raft_consensus.cc:385] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:41.987145  8349 raft_consensus.cc:740] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Initialized, Role: FOLLOWER
I20260709 13:02:41.988085  8349 consensus_queue.cc:260] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:41.990175  8349 ts_tablet_manager.cc:1434] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:02:41.991158  8349 tablet_bootstrap.cc:492] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7: Bootstrap starting.
I20260709 13:02:41.990089  8154 raft_consensus.cc:2468] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 1.
I20260709 13:02:41.997695  8350 tablet_bootstrap.cc:492] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3: No bootstrap required, opened a new log
I20260709 13:02:41.998181  8350 ts_tablet_manager.cc:1403] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.008s
I20260709 13:02:41.998286  8349 tablet_bootstrap.cc:654] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:42.001355  8350 raft_consensus.cc:359] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.002415  8350 raft_consensus.cc:385] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:42.002717  8350 raft_consensus.cc:740] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Initialized, Role: FOLLOWER
I20260709 13:02:42.003509  8350 consensus_queue.cc:260] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.005926  8350 ts_tablet_manager.cc:1434] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:02:42.006855  8350 tablet_bootstrap.cc:492] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3: Bootstrap starting.
I20260709 13:02:42.015504  8050 catalog_manager.cc:5697] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.021309  8050 catalog_manager.cc:5697] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.022776  8050 catalog_manager.cc:5697] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.028569  8349 tablet_bootstrap.cc:492] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7: No bootstrap required, opened a new log
I20260709 13:02:42.028995  8349 ts_tablet_manager.cc:1403] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.008s
I20260709 13:02:42.033376  8350 tablet_bootstrap.cc:654] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:42.033631  8349 raft_consensus.cc:359] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.043701  8349 raft_consensus.cc:385] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:42.044171  8349 raft_consensus.cc:740] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Initialized, Role: FOLLOWER
I20260709 13:02:42.044265  8389 raft_consensus.cc:493] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.045006  8389 raft_consensus.cc:515] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:42.045125  8349 consensus_queue.cc:260] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.047940  8349 ts_tablet_manager.cc:1434] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 13:02:42.048830  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:42.049507  8154 raft_consensus.cc:2393] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 788c1c01145a4d8e951fafe5cad72cd7 in current term 1: Already voted for candidate 9d7cd04e5edc4ade87478888855a363e in this term.
I20260709 13:02:42.051059  8389 leader_election.cc:290] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:42.051718  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:42.056275  8268 leader_election.cc:304] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7; no voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:42.057142  8389 raft_consensus.cc:3060] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:42.059945  8350 tablet_bootstrap.cc:492] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3: No bootstrap required, opened a new log
I20260709 13:02:42.060570  8350 ts_tablet_manager.cc:1403] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.006s
I20260709 13:02:42.063470  8350 raft_consensus.cc:359] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.062953  8052 catalog_manager.cc:5697] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.064280  8350 raft_consensus.cc:385] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:42.064680  8350 raft_consensus.cc:740] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Initialized, Role: FOLLOWER
I20260709 13:02:42.065299  8389 raft_consensus.cc:2749] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:42.065037  8052 catalog_manager.cc:5697] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.065624  8350 consensus_queue.cc:260] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.066926  8052 catalog_manager.cc:5697] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:42.067953  8350 ts_tablet_manager.cc:1434] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:02:42.068974  8350 tablet_bootstrap.cc:492] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3: Bootstrap starting.
I20260709 13:02:42.079224  8350 tablet_bootstrap.cc:654] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:42.085362  8350 tablet_bootstrap.cc:492] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3: No bootstrap required, opened a new log
I20260709 13:02:42.085896  8350 ts_tablet_manager.cc:1403] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.004s
I20260709 13:02:42.088801  8350 raft_consensus.cc:359] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.089579  8350 raft_consensus.cc:385] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:42.089978  8350 raft_consensus.cc:740] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Initialized, Role: FOLLOWER
I20260709 13:02:42.090703  8350 consensus_queue.cc:260] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.092891  8350 ts_tablet_manager.cc:1434] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:02:42.100100  8356 raft_consensus.cc:493] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.100592  8356 raft_consensus.cc:515] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.101517  8154 raft_consensus.cc:3060] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:42.102571  8356 leader_election.cc:290] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.103708  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:42.104110  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:42.104808  8231 raft_consensus.cc:2393] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 932d80a534fa4e7db50b464603e2b9e3 in current term 1: Already voted for candidate 788c1c01145a4d8e951fafe5cad72cd7 in this term.
I20260709 13:02:42.105933  8119 leader_election.cc:304] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3; no voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:42.108503  8356 raft_consensus.cc:2749] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:42.109360  8412 raft_consensus.cc:493] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.109231  8389 consensus_queue.cc:1048] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.109798  8412 raft_consensus.cc:515] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.111764  8412 leader_election.cc:290] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.112792  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:42.113379  8304 raft_consensus.cc:2468] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 0.
I20260709 13:02:42.114769  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:42.115051  8121 leader_election.cc:304] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:42.115381  8231 raft_consensus.cc:2468] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 0.
I20260709 13:02:42.115867  8412 raft_consensus.cc:2804] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:42.116218  8412 raft_consensus.cc:493] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.116530  8412 raft_consensus.cc:3060] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:42.124341  8382 consensus_queue.cc:1048] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:42.126623  8412 raft_consensus.cc:515] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.129032  8412 leader_election.cc:290] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 election: Requested vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.131021  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:42.131635  8304 raft_consensus.cc:3060] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:42.138083  8304 raft_consensus.cc:2468] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 1.
I20260709 13:02:42.141816  8121 leader_election.cc:304] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:42.142921  8412 raft_consensus.cc:2804] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:42.143430  8412 raft_consensus.cc:697] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 1 LEADER]: Becoming Leader. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:42.144127  8412 consensus_queue.cc:237] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.148001  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:42.148914  8231 raft_consensus.cc:3060] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:42.156914  8231 raft_consensus.cc:2468] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 1.
I20260709 13:02:42.159768  8354 raft_consensus.cc:493] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.160261  8354 raft_consensus.cc:515] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.162325  8354 leader_election.cc:290] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:42.163937  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932d80a534fa4e7db50b464603e2b9e3" is_pre_election: true
I20260709 13:02:42.164666  8154 raft_consensus.cc:2393] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 788c1c01145a4d8e951fafe5cad72cd7 in current term 1: Already voted for candidate 9d7cd04e5edc4ade87478888855a363e in this term.
I20260709 13:02:42.164383  8052 catalog_manager.cc:5697] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "932d80a534fa4e7db50b464603e2b9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.166383  8230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:42.167883  8268 leader_election.cc:304] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7; no voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:42.168649  8354 raft_consensus.cc:3060] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:42.175316  8354 raft_consensus.cc:2749] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:42.204411  8355 consensus_queue.cc:1048] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:42.215456  8355 consensus_queue.cc:1048] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.232339  8354 consensus_queue.cc:1048] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.242398  8154 raft_consensus.cc:3060] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:42.251837  8412 raft_consensus.cc:493] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.252481  8412 raft_consensus.cc:515] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.257596  8230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7cd04e5edc4ade87478888855a363e" is_pre_election: true
I20260709 13:02:42.257669  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7" is_pre_election: true
I20260709 13:02:42.259362  8121 leader_election.cc:304] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3; no voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:42.259047  8390 consensus_queue.cc:1048] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:42.260281  8356 raft_consensus.cc:2749] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:02:42.261468  8412 leader_election.cc:290] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.272571  8355 consensus_queue.cc:1048] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.290057  8391 consensus_queue.cc:1048] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:42.289978  8387 consensus_queue.cc:1048] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:42.304728  8154 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:42.316656  8391 consensus_queue.cc:1048] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.401346  8391 consensus_queue.cc:1048] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.412768  8387 consensus_queue.cc:1048] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.424986  8390 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:42.425827  8382 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.474398  8390 consensus_queue.cc:1048] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.486891  8354 consensus_queue.cc:1048] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.520620  8230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
mode: GRACEFUL
new_leader_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:35112
I20260709 13:02:42.521229  8230 raft_consensus.cc:606] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Received request to transfer leadership to TS 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:42.535581  8305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085"
dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
mode: GRACEFUL
new_leader_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:40890
I20260709 13:02:42.536151  8305 raft_consensus.cc:606] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Received request to transfer leadership to TS 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:42.540984  8230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
mode: GRACEFUL
new_leader_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:35112
I20260709 13:02:42.541489  8230 raft_consensus.cc:606] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Received request to transfer leadership to TS 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:42.546802  8230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56143748b832452c83027f7a0016323f"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
mode: GRACEFUL
new_leader_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:35112
I20260709 13:02:42.547379  8230 raft_consensus.cc:606] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Received request to transfer leadership to TS 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:42.555483  8305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5"
dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
mode: GRACEFUL
new_leader_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:40890
I20260709 13:02:42.556102  8305 raft_consensus.cc:606] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Received request to transfer leadership to TS 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:42.561550  8305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a"
dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
mode: GRACEFUL
new_leader_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:40890
I20260709 13:02:42.562249  8305 raft_consensus.cc:606] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Received request to transfer leadership to TS 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:42.576355  8354 raft_consensus.cc:993] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7: : Instructing follower 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:42.576885  8354 raft_consensus.cc:1081] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Signalling peer 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:42.579016  8231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:37408
I20260709 13:02:42.579257  8155 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
mode: GRACEFUL
new_leader_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:46016
I20260709 13:02:42.579571  8231 raft_consensus.cc:493] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.579862  8155 raft_consensus.cc:606] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 1 LEADER]: Received request to transfer leadership to TS 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:42.580070  8231 raft_consensus.cc:3060] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.586439  8231 raft_consensus.cc:515] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.588898  8231 leader_election.cc:290] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 2 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.589529  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:42.589932  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:42.590133  8305 raft_consensus.cc:3055] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.590538  8305 raft_consensus.cc:740] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:42.590602  8154 raft_consensus.cc:3060] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.591446  8305 consensus_queue.cc:260] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.592826  8305 raft_consensus.cc:3060] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.595942  8154 raft_consensus.cc:2468] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 2.
I20260709 13:02:42.597190  8197 leader_election.cc:304] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:42.597949  8387 raft_consensus.cc:2804] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.598448  8387 raft_consensus.cc:697] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:42.599429  8305 raft_consensus.cc:2468] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 2.
I20260709 13:02:42.599386  8387 consensus_queue.cc:237] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.604885  8355 raft_consensus.cc:993] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e: : Instructing follower 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:42.605341  8355 raft_consensus.cc:1081] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Signalling peer 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:42.607043  8154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:50880
I20260709 13:02:42.607585  8154 raft_consensus.cc:493] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.608022  8154 raft_consensus.cc:3060] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.607651  8050 catalog_manager.cc:5697] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 1 to 2, leader changed from 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67) to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.614378  8154 raft_consensus.cc:515] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:42.616465  8154 leader_election.cc:290] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 election: Requested vote from peers 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:42.617599  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:42.617810  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:42.618155  8305 raft_consensus.cc:3060] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.618384  8231 raft_consensus.cc:3055] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.618745  8231 raft_consensus.cc:740] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:42.619509  8231 consensus_queue.cc:260] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:42.620747  8231 raft_consensus.cc:3060] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.623732  8305 raft_consensus.cc:2468] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 2.
I20260709 13:02:42.624975  8121 leader_election.cc:304] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:42.625866  8412 raft_consensus.cc:2804] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.626410  8412 raft_consensus.cc:697] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:42.627197  8412 consensus_queue.cc:237] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:42.627767  8231 raft_consensus.cc:2468] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 2.
I20260709 13:02:42.634402  8050 catalog_manager.cc:5697] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 reported cstate change: term changed from 1 to 2, leader changed from 9d7cd04e5edc4ade87478888855a363e (127.7.213.66) to 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "932d80a534fa4e7db50b464603e2b9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.675191  8412 consensus_queue.cc:1048] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.682252  8356 raft_consensus.cc:993] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3: : Instructing follower 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:42.683038  8412 raft_consensus.cc:1081] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 1 LEADER]: Signalling peer 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:42.684769  8230 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:37398
I20260709 13:02:42.685324  8230 raft_consensus.cc:493] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.685688  8230 raft_consensus.cc:3060] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.689052  8356 consensus_queue.cc:1048] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:42.693053  8230 raft_consensus.cc:515] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.695282  8230 leader_election.cc:290] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 2 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.696678  8231 raft_consensus.cc:1240] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Rejecting Update request from peer 932d80a534fa4e7db50b464603e2b9e3 for earlier term 1. Current term is 2. Ops: []
I20260709 13:02:42.696761  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:42.697245  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:42.697448  8154 raft_consensus.cc:3055] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.697800  8154 raft_consensus.cc:740] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:42.698585  8154 consensus_queue.cc:260] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.699491  8154 raft_consensus.cc:3060] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.707824  8154 raft_consensus.cc:2468] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 2.
I20260709 13:02:42.709278  8197 leader_election.cc:304] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 788c1c01145a4d8e951fafe5cad72cd7
I20260709 13:02:42.710291  8355 raft_consensus.cc:2804] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.710857  8355 raft_consensus.cc:697] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:42.711791  8355 consensus_queue.cc:237] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.720559  8050 catalog_manager.cc:5697] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 1 to 2, leader changed from 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65) to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.774317  8382 raft_consensus.cc:993] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7: : Instructing follower 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:42.774799  8389 raft_consensus.cc:1081] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Signalling peer 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:42.777437  8154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:50868
I20260709 13:02:42.777927  8154 raft_consensus.cc:493] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.778318  8154 raft_consensus.cc:3060] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.782631  8154 raft_consensus.cc:515] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.785122  8230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:42.784734  8154 leader_election.cc:290] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 election: Requested vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.785267  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:42.785737  8230 raft_consensus.cc:3060] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.785838  8305 raft_consensus.cc:3055] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.786239  8305 raft_consensus.cc:740] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:42.787006  8305 consensus_queue.cc:260] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.788024  8305 raft_consensus.cc:3060] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.792362  8305 raft_consensus.cc:2468] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 2.
I20260709 13:02:42.792392  8230 raft_consensus.cc:2468] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 2.
I20260709 13:02:42.793519  8121 leader_election.cc:304] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:42.794358  8356 raft_consensus.cc:2804] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.794822  8356 raft_consensus.cc:697] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:42.795617  8356 consensus_queue.cc:237] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.803167  8049 catalog_manager.cc:5697] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 reported cstate change: term changed from 1 to 2, leader changed from 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67) to 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "932d80a534fa4e7db50b464603e2b9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.835836  8382 raft_consensus.cc:993] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7: : Instructing follower 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:42.836334  8389 raft_consensus.cc:1081] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Signalling peer 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:42.838002  8230 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:37408
I20260709 13:02:42.838603  8230 raft_consensus.cc:493] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.838953  8230 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.845470  8230 raft_consensus.cc:515] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.847687  8230 leader_election.cc:290] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 2 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.848817  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:42.849073  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:42.849435  8305 raft_consensus.cc:3055] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.849660  8154 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.849772  8305 raft_consensus.cc:740] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:42.850720  8305 consensus_queue.cc:260] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.851944  8305 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.854368  8154 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 2.
I20260709 13:02:42.855423  8197 leader_election.cc:304] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:42.856141  8355 raft_consensus.cc:2804] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.856563  8355 raft_consensus.cc:697] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:42.857362  8355 consensus_queue.cc:237] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.858454  8305 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 2.
I20260709 13:02:42.864063  8049 catalog_manager.cc:5697] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 1 to 2, leader changed from 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67) to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.867393  8387 raft_consensus.cc:993] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e: : Instructing follower 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:42.867887  8355 raft_consensus.cc:1081] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Signalling peer 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:42.870658  8154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56143748b832452c83027f7a0016323f"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:50880
I20260709 13:02:42.871289  8154 raft_consensus.cc:493] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.871651  8154 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.877651  8154 raft_consensus.cc:515] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.880707  8230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:42.881382  8230 raft_consensus.cc:3055] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.881704  8230 raft_consensus.cc:740] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:42.881930  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:42.882581  8305 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.882517  8230 consensus_queue.cc:260] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.880707  8154 leader_election.cc:290] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 election: Requested vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.883679  8230 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.891469  8230 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 2.
I20260709 13:02:42.891469  8305 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 2.
I20260709 13:02:42.892853  8119 leader_election.cc:304] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:42.893543  8356 raft_consensus.cc:2804] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.894037  8356 raft_consensus.cc:697] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:42.894860  8356 consensus_queue.cc:237] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.902200  8049 catalog_manager.cc:5697] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 reported cstate change: term changed from 1 to 2, leader changed from 9d7cd04e5edc4ade87478888855a363e (127.7.213.66) to 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "932d80a534fa4e7db50b464603e2b9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:42.940672  8387 raft_consensus.cc:993] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e: : Instructing follower 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:42.941195  8387 raft_consensus.cc:1081] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Signalling peer 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:42.943104  8154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:50880
I20260709 13:02:42.943610  8154 raft_consensus.cc:493] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:42.943984  8154 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.948173  8154 raft_consensus.cc:515] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.949687  8154 leader_election.cc:290] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 election: Requested vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:42.950525  8230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:42.951130  8230 raft_consensus.cc:3055] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:42.950909  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:42.951447  8230 raft_consensus.cc:740] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:42.951675  8305 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.952417  8230 consensus_queue.cc:260] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.953630  8230 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:42.958593  8305 raft_consensus.cc:2468] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 2.
I20260709 13:02:42.959968  8121 leader_election.cc:304] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:42.960520  8230 raft_consensus.cc:2468] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 2.
I20260709 13:02:42.960901  8356 raft_consensus.cc:2804] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:42.961458  8356 raft_consensus.cc:697] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Becoming Leader. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:42.962311  8356 consensus_queue.cc:237] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:42.969799  8049 catalog_manager.cc:5697] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 reported cstate change: term changed from 1 to 2, leader changed from 9d7cd04e5edc4ade87478888855a363e (127.7.213.66) to 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "932d80a534fa4e7db50b464603e2b9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:43.003816  8154 raft_consensus.cc:1275] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.005379  8355 consensus_queue.cc:1048] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.017619  8305 raft_consensus.cc:1275] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.019063  8355 consensus_queue.cc:1048] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.046803  8230 raft_consensus.cc:1275] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.048326  8356 consensus_queue.cc:1048] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:43.054481  8305 raft_consensus.cc:1275] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.056294  8438 consensus_queue.cc:1048] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.124912  8305 raft_consensus.cc:3060] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:43.129954  8305 raft_consensus.cc:1275] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.131312  8355 consensus_queue.cc:1048] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.140246  8154 raft_consensus.cc:1275] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.141834  8374 consensus_queue.cc:1048] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.255429  8305 raft_consensus.cc:1275] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.256795  8356 consensus_queue.cc:1048] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.265185  8230 raft_consensus.cc:1275] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.266927  8412 consensus_queue.cc:1048] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.290009  8154 raft_consensus.cc:1275] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.290194  8305 raft_consensus.cc:1275] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.291844  8387 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.292512  8374 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.343261  8230 raft_consensus.cc:1275] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.344839  8305 raft_consensus.cc:1275] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.344589  8438 consensus_queue.cc:1048] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.346072  8438 consensus_queue.cc:1048] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:43.451588  8305 raft_consensus.cc:1275] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.453154  8442 consensus_queue.cc:1048] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:43.463302  8230 raft_consensus.cc:1275] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:43.464952  8442 consensus_queue.cc:1048] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:45.616142  8021 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:02:45.617034  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:45.644004  8230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
mode: GRACEFUL
new_leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
 from {username='slave'} at 127.0.0.1:35124
I20260709 13:02:45.644564  8230 raft_consensus.cc:606] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Received request to transfer leadership to TS 788c1c01145a4d8e951fafe5cad72cd7
I20260709 13:02:45.657186  8154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56143748b832452c83027f7a0016323f"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
mode: GRACEFUL
new_leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
 from {username='slave'} at 127.0.0.1:46018
I20260709 13:02:45.657752  8154 raft_consensus.cc:606] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Received request to transfer leadership to TS 788c1c01145a4d8e951fafe5cad72cd7
I20260709 13:02:45.659065  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:02:45.659679  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:46.029973  8451 raft_consensus.cc:993] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3: : Instructing follower 788c1c01145a4d8e951fafe5cad72cd7 to start an election
I20260709 13:02:46.030426  8442 raft_consensus.cc:1081] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Signalling peer 788c1c01145a4d8e951fafe5cad72cd7 to start an election
I20260709 13:02:46.031894  8304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56143748b832452c83027f7a0016323f"
dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
 from {username='slave'} at 127.0.0.1:41418
I20260709 13:02:46.032434  8304 raft_consensus.cc:493] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:46.032778  8304 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.037806  8304 raft_consensus.cc:515] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.039631  8304 leader_election.cc:290] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 3 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:46.040859  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:46.040855  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:46.041720  8154 raft_consensus.cc:3055] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:46.041720  8231 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.042246  8154 raft_consensus.cc:740] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:46.043061  8154 consensus_queue.cc:260] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.044319  8154 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.049265  8231 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 3.
I20260709 13:02:46.050606  8268 leader_election.cc:304] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:46.050899  8154 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 3.
I20260709 13:02:46.051323  8389 raft_consensus.cc:2804] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:46.052645  8389 raft_consensus.cc:697] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Becoming Leader. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:46.053491  8389 consensus_queue.cc:237] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.063172  8050 catalog_manager.cc:5697] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 reported cstate change: term changed from 2 to 3, leader changed from 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65) to 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67). New cstate: current_term: 3 leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.088951  8458 raft_consensus.cc:993] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e: : Instructing follower 788c1c01145a4d8e951fafe5cad72cd7 to start an election
I20260709 13:02:46.089366  8387 raft_consensus.cc:1081] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Signalling peer 788c1c01145a4d8e951fafe5cad72cd7 to start an election
I20260709 13:02:46.090858  8304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5"
dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
 from {username='slave'} at 127.0.0.1:41426
I20260709 13:02:46.091410  8304 raft_consensus.cc:493] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:46.091722  8304 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.097121  8304 raft_consensus.cc:515] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.098950  8304 leader_election.cc:290] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 3 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:46.099857  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:46.099996  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:46.100438  8154 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.100617  8231 raft_consensus.cc:3055] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:46.100979  8231 raft_consensus.cc:740] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:46.101718  8231 consensus_queue.cc:260] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.102710  8231 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.104918  8154 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 3.
I20260709 13:02:46.106009  8271 leader_election.cc:304] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:46.106885  8389 raft_consensus.cc:2804] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:46.107302  8389 raft_consensus.cc:697] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Becoming Leader. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:46.108208  8231 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 3.
I20260709 13:02:46.108067  8389 consensus_queue.cc:237] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.115999  8050 catalog_manager.cc:5697] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 reported cstate change: term changed from 2 to 3, leader changed from 9d7cd04e5edc4ade87478888855a363e (127.7.213.66) to 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67). New cstate: current_term: 3 leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.461577  8154 raft_consensus.cc:1275] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Refusing update from remote peer 788c1c01145a4d8e951fafe5cad72cd7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:46.462944  8389 consensus_queue.cc:1048] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.472203  8231 raft_consensus.cc:1275] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Refusing update from remote peer 788c1c01145a4d8e951fafe5cad72cd7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:46.474272  8389 consensus_queue.cc:1048] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.602548  8154 raft_consensus.cc:1275] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Refusing update from remote peer 788c1c01145a4d8e951fafe5cad72cd7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:46.604135  8389 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.612958  8231 raft_consensus.cc:1275] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Refusing update from remote peer 788c1c01145a4d8e951fafe5cad72cd7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:46.614326  8466 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:46.660657  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:46.671506  8154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
mode: GRACEFUL
new_leader_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:46016
I20260709 13:02:46.672308  8154 raft_consensus.cc:606] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Received request to transfer leadership to TS 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:46.677521  8154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
mode: GRACEFUL
new_leader_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:46016
I20260709 13:02:46.678017  8154 raft_consensus.cc:606] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Received request to transfer leadership to TS 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:46.682580  8154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
mode: GRACEFUL
new_leader_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:46016
I20260709 13:02:46.683074  8154 raft_consensus.cc:606] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Received request to transfer leadership to TS 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:46.688511  8304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56143748b832452c83027f7a0016323f"
dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
mode: GRACEFUL
new_leader_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:40890
I20260709 13:02:46.689170  8304 raft_consensus.cc:606] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Received request to transfer leadership to TS 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:46.697186  8304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5"
dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
mode: GRACEFUL
new_leader_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:40890
I20260709 13:02:46.697705  8304 raft_consensus.cc:606] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Received request to transfer leadership to TS 9d7cd04e5edc4ade87478888855a363e
I20260709 13:02:46.781157  8468 raft_consensus.cc:993] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3: : Instructing follower 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:46.781654  8468 raft_consensus.cc:1081] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Signalling peer 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:46.783430  8231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:37398
I20260709 13:02:46.784121  8231 raft_consensus.cc:493] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:46.784564  8231 raft_consensus.cc:3060] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.789816  8231 raft_consensus.cc:515] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:46.791636  8231 leader_election.cc:290] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 3 election: Requested vote from peers 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329), 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991)
I20260709 13:02:46.792781  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:46.792693  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ee3fb267edc49bf838bac8d18d65d14" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:46.793471  8154 raft_consensus.cc:3055] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:46.793560  8304 raft_consensus.cc:3060] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.793895  8154 raft_consensus.cc:740] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:46.794691  8154 consensus_queue.cc:260] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:46.795958  8154 raft_consensus.cc:3060] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.800956  8304 raft_consensus.cc:2468] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 3.
I20260709 13:02:46.802585  8154 raft_consensus.cc:2468] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 3.
I20260709 13:02:46.802466  8196 leader_election.cc:304] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:46.803601  8387 raft_consensus.cc:2804] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:46.804930  8387 raft_consensus.cc:697] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 3 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:46.805799  8387 consensus_queue.cc:237] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } }
I20260709 13:02:46.813056  8050 catalog_manager.cc:5697] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 2 to 3, leader changed from 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65) to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 3 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:46.857322  8442 raft_consensus.cc:993] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3: : Instructing follower 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:46.857740  8468 raft_consensus.cc:1081] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Signalling peer 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:46.859236  8231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:37398
I20260709 13:02:46.860023  8231 raft_consensus.cc:493] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:46.860469  8231 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.865074  8231 raft_consensus.cc:515] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.866598  8231 leader_election.cc:290] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 3 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:46.867609  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:46.867674  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:46.868233  8154 raft_consensus.cc:3055] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:46.868299  8304 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.868502  8154 raft_consensus.cc:740] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:46.869222  8154 consensus_queue.cc:260] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.870013  8154 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.873099  8304 raft_consensus.cc:2468] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 3.
I20260709 13:02:46.873971  8154 raft_consensus.cc:2468] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 3.
I20260709 13:02:46.874328  8196 leader_election.cc:304] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:46.875139  8387 raft_consensus.cc:2804] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:46.875540  8387 raft_consensus.cc:697] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 3 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:46.876286  8387 consensus_queue.cc:237] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.882664  8050 catalog_manager.cc:5697] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 2 to 3, leader changed from 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65) to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 3 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.898841  8442 raft_consensus.cc:993] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3: : Instructing follower 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:46.899397  8468 raft_consensus.cc:1081] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Signalling peer 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:46.901083  8231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:37398
I20260709 13:02:46.901690  8231 raft_consensus.cc:493] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:46.902020  8231 raft_consensus.cc:3060] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.907163  8231 raft_consensus.cc:515] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.908885  8231 leader_election.cc:290] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 3 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:46.909787  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:46.909961  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa4c1171ee57481880d302faecbfb085" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:46.910351  8154 raft_consensus.cc:3055] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:46.910437  8304 raft_consensus.cc:3060] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.910698  8154 raft_consensus.cc:740] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:46.911434  8154 consensus_queue.cc:260] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.912545  8154 raft_consensus.cc:3060] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:46.916810  8304 raft_consensus.cc:2468] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 3.
I20260709 13:02:46.917974  8196 leader_election.cc:304] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:46.918270  8154 raft_consensus.cc:2468] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 3.
I20260709 13:02:46.918857  8387 raft_consensus.cc:2804] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:46.919381  8387 raft_consensus.cc:697] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 3 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:46.920701  8387 consensus_queue.cc:237] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.921947  8452 raft_consensus.cc:993] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7: : Instructing follower 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:46.922413  8466 raft_consensus.cc:1081] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Signalling peer 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:46.924045  8231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56143748b832452c83027f7a0016323f"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:37408
I20260709 13:02:46.924798  8231 raft_consensus.cc:493] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:46.925201  8231 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:46.929329  8050 catalog_manager.cc:5697] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 2 to 3, leader changed from 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65) to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 3 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:46.931181  8231 raft_consensus.cc:515] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.932911  8231 leader_election.cc:290] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 4 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:46.933852  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:46.934425  8154 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:46.934324  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56143748b832452c83027f7a0016323f" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:46.935014  8304 raft_consensus.cc:3055] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:46.935393  8304 raft_consensus.cc:740] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:46.936259  8304 consensus_queue.cc:260] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.937526  8304 raft_consensus.cc:3060] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:46.941020  8154 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 4.
I20260709 13:02:46.942044  8197 leader_election.cc:304] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:46.942873  8387 raft_consensus.cc:2804] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:46.943354  8387 raft_consensus.cc:697] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 4 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:46.943899  8304 raft_consensus.cc:2468] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 4.
I20260709 13:02:46.944260  8387 consensus_queue.cc:237] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:46.950395  8050 catalog_manager.cc:5697] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 3 to 4, leader changed from 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67) to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 4 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.231961  8452 raft_consensus.cc:993] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7: : Instructing follower 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:47.232450  8466 raft_consensus.cc:1081] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Signalling peer 9d7cd04e5edc4ade87478888855a363e to start an election
I20260709 13:02:47.234256  8231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
 from {username='slave'} at 127.0.0.1:37408
I20260709 13:02:47.234793  8231 raft_consensus.cc:493] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:47.235203  8231 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:47.239357  8231 raft_consensus.cc:515] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:47.240407  8305 raft_consensus.cc:1275] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:47.241763  8231 leader_election.cc:290] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 4 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:47.241681  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:47.242298  8305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "9d7cd04e5edc4ade87478888855a363e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:47.242305  8154 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:47.242897  8305 raft_consensus.cc:3055] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:47.243182  8305 raft_consensus.cc:740] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:47.242511  8387 consensus_queue.cc:1048] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:02:47.243842  8305 consensus_queue.cc:260] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:47.244961  8305 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:47.248260  8154 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 4.
I20260709 13:02:47.249631  8197 leader_election.cc:304] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 932d80a534fa4e7db50b464603e2b9e3, 9d7cd04e5edc4ade87478888855a363e; no voters: 
I20260709 13:02:47.250442  8305 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7cd04e5edc4ade87478888855a363e in term 4.
I20260709 13:02:47.250842  8387 raft_consensus.cc:2804] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:47.252131  8387 raft_consensus.cc:697] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 4 LEADER]: Becoming Leader. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:47.253042  8387 consensus_queue.cc:237] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:47.260447  8154 raft_consensus.cc:1275] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:47.262320  8387 consensus_queue.cc:1048] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:47.262676  8050 catalog_manager.cc:5697] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e reported cstate change: term changed from 3 to 4, leader changed from 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67) to 9d7cd04e5edc4ade87478888855a363e (127.7.213.66). New cstate: current_term: 4 leader_uuid: "9d7cd04e5edc4ade87478888855a363e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:47.294783  8305 raft_consensus.cc:1275] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:47.296226  8387 consensus_queue.cc:1048] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:47.305727  8154 raft_consensus.cc:1275] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:47.307013  8477 consensus_queue.cc:1048] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:47.393232  8154 raft_consensus.cc:1275] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:47.394510  8477 consensus_queue.cc:1048] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:47.402468  8305 raft_consensus.cc:1275] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:47.403947  8477 consensus_queue.cc:1048] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:47.475672  8305 raft_consensus.cc:1275] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:47.477396  8477 consensus_queue.cc:1048] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:47.486656  8154 raft_consensus.cc:1275] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:47.488183  8485 consensus_queue.cc:1048] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:47.767405  8154 raft_consensus.cc:1275] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:47.768870  8387 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:47.777798  8304 raft_consensus.cc:1275] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Refusing update from remote peer 9d7cd04e5edc4ade87478888855a363e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:47.779485  8485 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:49.707983  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:49.715205  8231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
mode: GRACEFUL
new_leader_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:35124
I20260709 13:02:49.715961  8231 raft_consensus.cc:606] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 4 LEADER]: Received request to transfer leadership to TS 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:49.718816  8231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
mode: GRACEFUL
new_leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
 from {username='slave'} at 127.0.0.1:35124
I20260709 13:02:49.719323  8231 raft_consensus.cc:606] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Received request to transfer leadership to TS 788c1c01145a4d8e951fafe5cad72cd7
I20260709 13:02:49.721859  8231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
mode: GRACEFUL
new_leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
 from {username='slave'} at 127.0.0.1:35124
I20260709 13:02:49.722334  8231 raft_consensus.cc:606] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Received request to transfer leadership to TS 788c1c01145a4d8e951fafe5cad72cd7
I20260709 13:02:49.725183  8231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
mode: GRACEFUL
new_leader_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:35124
I20260709 13:02:49.725638  8231 raft_consensus.cc:606] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Received request to transfer leadership to TS 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:49.728091  8231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5"
dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
mode: GRACEFUL
new_leader_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:35124
I20260709 13:02:49.728504  8231 raft_consensus.cc:606] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 3 LEADER]: Received request to transfer leadership to TS 932d80a534fa4e7db50b464603e2b9e3
I20260709 13:02:49.729651  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:49.730123  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:49.749969  8477 raft_consensus.cc:993] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e: : Instructing follower 788c1c01145a4d8e951fafe5cad72cd7 to start an election
I20260709 13:02:49.750481  8491 raft_consensus.cc:1081] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Signalling peer 788c1c01145a4d8e951fafe5cad72cd7 to start an election
I20260709 13:02:49.752202  8304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3"
dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
 from {username='slave'} at 127.0.0.1:41426
I20260709 13:02:49.752694  8304 raft_consensus.cc:493] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:49.753072  8304 raft_consensus.cc:3060] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.758026  8304 raft_consensus.cc:515] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:49.759855  8304 leader_election.cc:290] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 2 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:49.760967  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:49.760978  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53ddf1f9a83248d1bb76ce958ca6fce3" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:49.761785  8231 raft_consensus.cc:3055] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:49.761785  8154 raft_consensus.cc:3060] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.762072  8231 raft_consensus.cc:740] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:49.762797  8231 consensus_queue.cc:260] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:49.763711  8231 raft_consensus.cc:3060] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:49.766707  8154 raft_consensus.cc:2468] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 2.
I20260709 13:02:49.768069  8271 leader_election.cc:304] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:49.768678  8231 raft_consensus.cc:2468] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 2.
I20260709 13:02:49.768893  8492 raft_consensus.cc:2804] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:49.770017  8492 raft_consensus.cc:697] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 LEADER]: Becoming Leader. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:49.771064  8492 consensus_queue.cc:237] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:49.777650  8052 catalog_manager.cc:5697] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 reported cstate change: term changed from 1 to 2, leader changed from 9d7cd04e5edc4ade87478888855a363e (127.7.213.66) to 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:49.964061  8491 raft_consensus.cc:993] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e: : Instructing follower 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:49.964510  8477 raft_consensus.cc:1081] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 4 LEADER]: Signalling peer 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:49.966063  8154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:50880
I20260709 13:02:49.966509  8154 raft_consensus.cc:493] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:49.966851  8154 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.972071  8154 raft_consensus.cc:515] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:49.974320  8154 leader_election.cc:290] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 5 election: Requested vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:49.975090  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:49.975461  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013e2fecd579459c92f501140d58c5a5" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:49.975653  8231 raft_consensus.cc:3055] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:02:49.975977  8231 raft_consensus.cc:740] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:49.975977  8304 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.976816  8231 consensus_queue.cc:260] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:49.977947  8231 raft_consensus.cc:3060] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:02:49.982228  8304 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 5.
I20260709 13:02:49.983603  8121 leader_election.cc:304] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:49.983721  8231 raft_consensus.cc:2468] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 5.
I20260709 13:02:49.985772  8499 raft_consensus.cc:2804] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:02:49.986816  8499 raft_consensus.cc:697] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 5 LEADER]: Becoming Leader. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:49.987527  8499 consensus_queue.cc:237] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:49.994737  8052 catalog_manager.cc:5697] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 reported cstate change: term changed from 4 to 5, leader changed from 9d7cd04e5edc4ade87478888855a363e (127.7.213.66) to 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65). New cstate: current_term: 5 leader_uuid: "932d80a534fa4e7db50b464603e2b9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.046687  8491 raft_consensus.cc:993] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e: : Instructing follower 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:50.047189  8477 raft_consensus.cc:1081] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Signalling peer 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:50.048765  8154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:50880
I20260709 13:02:50.049244  8154 raft_consensus.cc:493] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:50.049530  8154 raft_consensus.cc:3060] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:50.053871  8154 raft_consensus.cc:515] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:50.055491  8154 leader_election.cc:290] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 3 election: Requested vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:50.056341  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:50.056569  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93ac48387e5a4acabd3aedc7f2a6270a" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:50.056845  8231 raft_consensus.cc:3055] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:50.057071  8304 raft_consensus.cc:3060] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:50.057192  8231 raft_consensus.cc:740] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:50.058014  8231 consensus_queue.cc:260] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:50.058871  8231 raft_consensus.cc:3060] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:50.061669  8304 raft_consensus.cc:2468] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 3.
I20260709 13:02:50.062783  8121 leader_election.cc:304] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:50.063292  8231 raft_consensus.cc:2468] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 3.
I20260709 13:02:50.063644  8499 raft_consensus.cc:2804] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:50.064172  8499 raft_consensus.cc:697] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 3 LEADER]: Becoming Leader. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:50.064956  8499 consensus_queue.cc:237] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:50.071156  8052 catalog_manager.cc:5697] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 reported cstate change: term changed from 2 to 3, leader changed from 9d7cd04e5edc4ade87478888855a363e (127.7.213.66) to 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "932d80a534fa4e7db50b464603e2b9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.119601  8491 raft_consensus.cc:993] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e: : Instructing follower 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:50.120047  8477 raft_consensus.cc:1081] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 3 LEADER]: Signalling peer 932d80a534fa4e7db50b464603e2b9e3 to start an election
I20260709 13:02:50.121665  8154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5"
dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
 from {username='slave'} at 127.0.0.1:50880
I20260709 13:02:50.122231  8154 raft_consensus.cc:493] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:50.122562  8154 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:50.126924  8154 raft_consensus.cc:515] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:50.128446  8154 leader_election.cc:290] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 4 election: Requested vote from peers 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645), 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67:33329)
I20260709 13:02:50.129279  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:50.129408  8304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6fb6e3565064bd1a87124ed6b20c3d5" candidate_uuid: "932d80a534fa4e7db50b464603e2b9e3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
I20260709 13:02:50.129841  8231 raft_consensus.cc:3055] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:50.129842  8304 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:50.130287  8231 raft_consensus.cc:740] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:50.131000  8231 consensus_queue.cc:260] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:50.132058  8231 raft_consensus.cc:3060] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:50.135986  8304 raft_consensus.cc:2468] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 4.
I20260709 13:02:50.137104  8121 leader_election.cc:304] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:50.137974  8499 raft_consensus.cc:2804] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:50.138049  8231 raft_consensus.cc:2468] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 932d80a534fa4e7db50b464603e2b9e3 in term 4.
I20260709 13:02:50.138531  8499 raft_consensus.cc:697] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 4 LEADER]: Becoming Leader. State: Replica: 932d80a534fa4e7db50b464603e2b9e3, State: Running, Role: LEADER
I20260709 13:02:50.139384  8499 consensus_queue.cc:237] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:50.146772  8052 catalog_manager.cc:5697] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 reported cstate change: term changed from 3 to 4, leader changed from 9d7cd04e5edc4ade87478888855a363e (127.7.213.66) to 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65). New cstate: current_term: 4 leader_uuid: "932d80a534fa4e7db50b464603e2b9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:50.147886  8491 raft_consensus.cc:993] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e: : Instructing follower 788c1c01145a4d8e951fafe5cad72cd7 to start an election
I20260709 13:02:50.148362  8477 raft_consensus.cc:1081] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Signalling peer 788c1c01145a4d8e951fafe5cad72cd7 to start an election
I20260709 13:02:50.150044  8304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e"
dest_uuid: "788c1c01145a4d8e951fafe5cad72cd7"
 from {username='slave'} at 127.0.0.1:41426
I20260709 13:02:50.150506  8304 raft_consensus.cc:493] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:50.150801  8304 raft_consensus.cc:3060] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:50.156168  8304 raft_consensus.cc:515] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:50.157621  8304 leader_election.cc:290] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 3 election: Requested vote from peers 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991), 9d7cd04e5edc4ade87478888855a363e (127.7.213.66:35645)
I20260709 13:02:50.158699  8231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9d7cd04e5edc4ade87478888855a363e"
I20260709 13:02:50.158727  8154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "903a6b1533f64dd9afe641c52fc62c9e" candidate_uuid: "788c1c01145a4d8e951fafe5cad72cd7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "932d80a534fa4e7db50b464603e2b9e3"
I20260709 13:02:50.159382  8231 raft_consensus.cc:3055] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:50.159482  8154 raft_consensus.cc:3060] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:50.159713  8231 raft_consensus.cc:740] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9d7cd04e5edc4ade87478888855a363e, State: Running, Role: LEADER
I20260709 13:02:50.160454  8231 consensus_queue.cc:260] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:50.161299  8231 raft_consensus.cc:3060] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:50.164151  8154 raft_consensus.cc:2468] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 3.
I20260709 13:02:50.165189  8271 leader_election.cc:304] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788c1c01145a4d8e951fafe5cad72cd7, 932d80a534fa4e7db50b464603e2b9e3; no voters: 
I20260709 13:02:50.165864  8231 raft_consensus.cc:2468] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788c1c01145a4d8e951fafe5cad72cd7 in term 3.
I20260709 13:02:50.166065  8492 raft_consensus.cc:2804] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:50.166651  8492 raft_consensus.cc:697] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Becoming Leader. State: Replica: 788c1c01145a4d8e951fafe5cad72cd7, State: Running, Role: LEADER
I20260709 13:02:50.167539  8492 consensus_queue.cc:237] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } }
I20260709 13:02:50.174860  8052 catalog_manager.cc:5697] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 reported cstate change: term changed from 2 to 3, leader changed from 9d7cd04e5edc4ade87478888855a363e (127.7.213.66) to 788c1c01145a4d8e951fafe5cad72cd7 (127.7.213.67). New cstate: current_term: 3 leader_uuid: "788c1c01145a4d8e951fafe5cad72cd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:50.219125  8231 raft_consensus.cc:1275] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Refusing update from remote peer 788c1c01145a4d8e951fafe5cad72cd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:50.220584  8492 consensus_queue.cc:1048] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:50.233268  8154 raft_consensus.cc:1275] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Refusing update from remote peer 788c1c01145a4d8e951fafe5cad72cd7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:50.234745  8507 consensus_queue.cc:1048] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:02:50.370098  8231 raft_consensus.cc:1275] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 5 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:50.371452  8499 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:50.381088  8304 raft_consensus.cc:1275] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 5 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:02:50.383078  8512 consensus_queue.cc:1048] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:50.510358  8231 raft_consensus.cc:1275] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:50.511621  8499 consensus_queue.cc:1048] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:50.520804  8304 raft_consensus.cc:1275] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:50.522619  8499 consensus_queue.cc:1048] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:50.553588  8304 raft_consensus.cc:1275] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:50.555032  8499 consensus_queue.cc:1048] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "788c1c01145a4d8e951fafe5cad72cd7" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:50.566461  8231 raft_consensus.cc:1275] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Refusing update from remote peer 932d80a534fa4e7db50b464603e2b9e3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:50.568111  8517 consensus_queue.cc:1048] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:50.575867  8230 raft_consensus.cc:1275] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Refusing update from remote peer 788c1c01145a4d8e951fafe5cad72cd7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:50.577155  8492 consensus_queue.cc:1048] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7cd04e5edc4ade87478888855a363e" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:50.586302  8154 raft_consensus.cc:1275] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Refusing update from remote peer 788c1c01145a4d8e951fafe5cad72cd7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:50.588110  8493 consensus_queue.cc:1048] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "932d80a534fa4e7db50b464603e2b9e3" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:50.731137  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:50.736625  8021 tablet_server.cc:179] TabletServer@127.7.213.65:0 shutting down...
I20260709 13:02:50.767987  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:50.768769  8021 tablet_replica.cc:333] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3: stopping tablet replica
I20260709 13:02:50.769826  8021 raft_consensus.cc:2243] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:50.770750  8021 raft_consensus.cc:2272] T 93ac48387e5a4acabd3aedc7f2a6270a P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.773707  8021 tablet_replica.cc:333] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3: stopping tablet replica
I20260709 13:02:50.774232  8021 raft_consensus.cc:2243] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.774679  8021 raft_consensus.cc:2272] T 1e61de98e1e1405c8c9bade40e860281 P 932d80a534fa4e7db50b464603e2b9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.776551  8021 tablet_replica.cc:333] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3: stopping tablet replica
I20260709 13:02:50.777082  8021 raft_consensus.cc:2243] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.777506  8021 raft_consensus.cc:2272] T aa4c1171ee57481880d302faecbfb085 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.779208  8021 tablet_replica.cc:333] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3: stopping tablet replica
I20260709 13:02:50.779690  8021 raft_consensus.cc:2243] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:02:50.780485  8021 raft_consensus.cc:2272] T 013e2fecd579459c92f501140d58c5a5 P 932d80a534fa4e7db50b464603e2b9e3 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.782771  8021 tablet_replica.cc:333] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3: stopping tablet replica
I20260709 13:02:50.783504  8021 raft_consensus.cc:2243] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.783970  8021 raft_consensus.cc:2272] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 932d80a534fa4e7db50b464603e2b9e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.786080  8021 tablet_replica.cc:333] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3: stopping tablet replica
I20260709 13:02:50.786558  8021 raft_consensus.cc:2243] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:02:50.787274  8021 raft_consensus.cc:2272] T c6fb6e3565064bd1a87124ed6b20c3d5 P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.789377  8021 tablet_replica.cc:333] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3: stopping tablet replica
I20260709 13:02:50.789840  8021 raft_consensus.cc:2243] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.790221  8021 raft_consensus.cc:2272] T 903a6b1533f64dd9afe641c52fc62c9e P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.792240  8021 tablet_replica.cc:333] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3: stopping tablet replica
I20260709 13:02:50.792682  8021 raft_consensus.cc:2243] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.793140  8021 raft_consensus.cc:2272] T 3ee3fb267edc49bf838bac8d18d65d14 P 932d80a534fa4e7db50b464603e2b9e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:50.794862  8021 tablet_replica.cc:333] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3: stopping tablet replica
I20260709 13:02:50.795272  8021 raft_consensus.cc:2243] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:50.795727  8021 raft_consensus.cc:2272] T 56143748b832452c83027f7a0016323f P 932d80a534fa4e7db50b464603e2b9e3 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:02:50.950032  8271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.213.65:38991: connect: Connection refused (error 111)
I20260709 13:02:50.952390  8021 tablet_server.cc:196] TabletServer@127.7.213.65:0 shutdown complete.
W20260709 13:02:50.955338  8197 consensus_peers.cc:597] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e -> Peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Couldn't send request to peer 932d80a534fa4e7db50b464603e2b9e3. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:38991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:50.955791  8197 consensus_peers.cc:597] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e -> Peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Couldn't send request to peer 932d80a534fa4e7db50b464603e2b9e3. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:38991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:50.956208  8197 consensus_peers.cc:597] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e -> Peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Couldn't send request to peer 932d80a534fa4e7db50b464603e2b9e3. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:38991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:50.957494  8271 consensus_peers.cc:597] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 -> Peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Couldn't send request to peer 932d80a534fa4e7db50b464603e2b9e3. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:38991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:50.971716  8021 tablet_server.cc:179] TabletServer@127.7.213.66:0 shutting down...
I20260709 13:02:51.001122  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:51.001730  8021 tablet_replica.cc:333] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e: stopping tablet replica
I20260709 13:02:51.002439  8021 raft_consensus.cc:2243] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.002885  8021 raft_consensus.cc:2272] T 1e61de98e1e1405c8c9bade40e860281 P 9d7cd04e5edc4ade87478888855a363e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.004705  8021 tablet_replica.cc:333] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e: stopping tablet replica
I20260709 13:02:51.005263  8021 raft_consensus.cc:2243] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.005661  8021 raft_consensus.cc:2272] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 9d7cd04e5edc4ade87478888855a363e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.007735  8021 tablet_replica.cc:333] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e: stopping tablet replica
I20260709 13:02:51.008301  8021 raft_consensus.cc:2243] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:51.009016  8021 raft_consensus.cc:2272] T 3ee3fb267edc49bf838bac8d18d65d14 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.010533  8021 tablet_replica.cc:333] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e: stopping tablet replica
I20260709 13:02:51.011010  8021 raft_consensus.cc:2243] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.011389  8021 raft_consensus.cc:2272] T 903a6b1533f64dd9afe641c52fc62c9e P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.013383  8021 tablet_replica.cc:333] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e: stopping tablet replica
I20260709 13:02:51.013868  8021 raft_consensus.cc:2243] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.014252  8021 raft_consensus.cc:2272] T c6fb6e3565064bd1a87124ed6b20c3d5 P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.016152  8021 tablet_replica.cc:333] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e: stopping tablet replica
I20260709 13:02:51.016615  8021 raft_consensus.cc:2243] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:02:51.017401  8021 raft_consensus.cc:2272] T 56143748b832452c83027f7a0016323f P 9d7cd04e5edc4ade87478888855a363e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.018858  8021 tablet_replica.cc:333] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e: stopping tablet replica
I20260709 13:02:51.019289  8021 raft_consensus.cc:2243] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.019661  8021 raft_consensus.cc:2272] T 93ac48387e5a4acabd3aedc7f2a6270a P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.021476  8021 tablet_replica.cc:333] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e: stopping tablet replica
I20260709 13:02:51.021922  8021 raft_consensus.cc:2243] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.022351  8021 raft_consensus.cc:2272] T 013e2fecd579459c92f501140d58c5a5 P 9d7cd04e5edc4ade87478888855a363e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.024212  8021 tablet_replica.cc:333] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e: stopping tablet replica
I20260709 13:02:51.024633  8021 raft_consensus.cc:2243] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:51.025336  8021 raft_consensus.cc:2272] T aa4c1171ee57481880d302faecbfb085 P 9d7cd04e5edc4ade87478888855a363e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.054198  8021 tablet_server.cc:196] TabletServer@127.7.213.66:0 shutdown complete.
W20260709 13:02:51.054598  8271 consensus_peers.cc:597] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 -> Peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Couldn't send request to peer 932d80a534fa4e7db50b464603e2b9e3. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:38991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:51.067472  8271 consensus_peers.cc:597] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 -> Peer 932d80a534fa4e7db50b464603e2b9e3 (127.7.213.65:38991): Couldn't send request to peer 932d80a534fa4e7db50b464603e2b9e3. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:38991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:51.076968  8021 tablet_server.cc:179] TabletServer@127.7.213.67:0 shutting down...
I20260709 13:02:51.102849  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:02:51.103798  8021 tablet_replica.cc:333] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7: stopping tablet replica
I20260709 13:02:51.104434  8021 raft_consensus.cc:2243] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:02:51.105265  8021 raft_consensus.cc:2272] T 53ddf1f9a83248d1bb76ce958ca6fce3 P 788c1c01145a4d8e951fafe5cad72cd7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.108035  8021 tablet_replica.cc:333] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7: stopping tablet replica
I20260709 13:02:51.108538  8021 raft_consensus.cc:2243] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.108958  8021 raft_consensus.cc:2272] T 3ee3fb267edc49bf838bac8d18d65d14 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.110805  8021 tablet_replica.cc:333] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7: stopping tablet replica
I20260709 13:02:51.111254  8021 raft_consensus.cc:2243] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:51.112011  8021 raft_consensus.cc:2272] T 1e61de98e1e1405c8c9bade40e860281 P 788c1c01145a4d8e951fafe5cad72cd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.113484  8021 tablet_replica.cc:333] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7: stopping tablet replica
I20260709 13:02:51.113910  8021 raft_consensus.cc:2243] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.114217  8021 raft_consensus.cc:2272] T 56143748b832452c83027f7a0016323f P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.115653  8021 tablet_replica.cc:333] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7: stopping tablet replica
I20260709 13:02:51.116114  8021 raft_consensus.cc:2243] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:02:51.116736  8021 raft_consensus.cc:2272] T 903a6b1533f64dd9afe641c52fc62c9e P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.118963  8021 tablet_replica.cc:333] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7: stopping tablet replica
I20260709 13:02:51.119428  8021 raft_consensus.cc:2243] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.119879  8021 raft_consensus.cc:2272] T c6fb6e3565064bd1a87124ed6b20c3d5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.121721  8021 tablet_replica.cc:333] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7: stopping tablet replica
I20260709 13:02:51.122124  8021 raft_consensus.cc:2243] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.122452  8021 raft_consensus.cc:2272] T 93ac48387e5a4acabd3aedc7f2a6270a P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.124280  8021 tablet_replica.cc:333] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7: stopping tablet replica
I20260709 13:02:51.124684  8021 raft_consensus.cc:2243] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.125015  8021 raft_consensus.cc:2272] T 013e2fecd579459c92f501140d58c5a5 P 788c1c01145a4d8e951fafe5cad72cd7 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.126865  8021 tablet_replica.cc:333] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7: stopping tablet replica
I20260709 13:02:51.127250  8021 raft_consensus.cc:2243] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:02:51.127599  8021 raft_consensus.cc:2272] T aa4c1171ee57481880d302faecbfb085 P 788c1c01145a4d8e951fafe5cad72cd7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.153182  8021 tablet_server.cc:196] TabletServer@127.7.213.67:0 shutdown complete.
I20260709 13:02:51.171351  8021 master.cc:562] Master@127.7.213.126:34287 shutting down...
I20260709 13:02:51.310931  8021 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:02:51.311512  8021 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:02:51.311898  8021 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1951c9338083445db62cf3b5bfa3ff07: stopping tablet replica
I20260709 13:02:51.330861  8021 master.cc:584] Master@127.7.213.126:34287 shutdown complete.
788c1c01145a4d8e951fafe5cad72cd7  1
932d80a534fa4e7db50b464603e2b9e3  4
9d7cd04e5edc4ade87478888855a363e  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12990 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:02:51.370180  8021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.213.126:40949
I20260709 13:02:51.371341  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:51.377215  8525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:51.377216  8526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:51.379047  8528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:51.379477  8021 server_base.cc:1061] running on GCE node
I20260709 13:02:51.380465  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:51.380722  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:51.380899  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602171380882 us; error 0 us; skew 500 ppm
I20260709 13:02:51.381433  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:51.383997  8021 webserver.cc:533] Webserver started at http://127.7.213.126:43585/ using document root <none> and password file <none>
I20260709 13:02:51.384498  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:51.384702  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:51.384984  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:51.386112  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/master-0-root/instance:
uuid: "76d72720e986447b8a0e2fb653dda6e9"
format_stamp: "Formatted at 2026-07-09 13:02:51 on dist-test-slave-9g29"
I20260709 13:02:51.390599  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:02:51.394086  8533 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.395020  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:02:51.395290  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/master-0-root
uuid: "76d72720e986447b8a0e2fb653dda6e9"
format_stamp: "Formatted at 2026-07-09 13:02:51 on dist-test-slave-9g29"
I20260709 13:02:51.395539  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:51.412935  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:51.414091  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:51.453157  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.126:40949
I20260709 13:02:51.453256  8584 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.126:40949 every 8 connection(s)
I20260709 13:02:51.457219  8585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:51.466485  8585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9: Bootstrap starting.
I20260709 13:02:51.470913  8585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:51.475147  8585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9: No bootstrap required, opened a new log
I20260709 13:02:51.477347  8585 raft_consensus.cc:359] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d72720e986447b8a0e2fb653dda6e9" member_type: VOTER }
I20260709 13:02:51.477725  8585 raft_consensus.cc:385] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:51.478019  8585 raft_consensus.cc:740] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d72720e986447b8a0e2fb653dda6e9, State: Initialized, Role: FOLLOWER
I20260709 13:02:51.478583  8585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d72720e986447b8a0e2fb653dda6e9" member_type: VOTER }
I20260709 13:02:51.479033  8585 raft_consensus.cc:399] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:02:51.479247  8585 raft_consensus.cc:493] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:02:51.479570  8585 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:51.483541  8585 raft_consensus.cc:515] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d72720e986447b8a0e2fb653dda6e9" member_type: VOTER }
I20260709 13:02:51.484200  8585 leader_election.cc:304] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76d72720e986447b8a0e2fb653dda6e9; no voters: 
I20260709 13:02:51.485407  8585 leader_election.cc:290] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:02:51.485831  8588 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:51.487123  8588 raft_consensus.cc:697] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 1 LEADER]: Becoming Leader. State: Replica: 76d72720e986447b8a0e2fb653dda6e9, State: Running, Role: LEADER
I20260709 13:02:51.487869  8588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d72720e986447b8a0e2fb653dda6e9" member_type: VOTER }
I20260709 13:02:51.488348  8585 sys_catalog.cc:565] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:02:51.492623  8590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76d72720e986447b8a0e2fb653dda6e9. Latest consensus state: current_term: 1 leader_uuid: "76d72720e986447b8a0e2fb653dda6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d72720e986447b8a0e2fb653dda6e9" member_type: VOTER } }
I20260709 13:02:51.494158  8590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:51.493491  8589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76d72720e986447b8a0e2fb653dda6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d72720e986447b8a0e2fb653dda6e9" member_type: VOTER } }
I20260709 13:02:51.494839  8589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [sys.catalog]: This master's current role is: LEADER
I20260709 13:02:51.496209  8595 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:02:51.501381  8595 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:02:51.504839  8021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:02:51.510236  8595 catalog_manager.cc:1383] Generated new cluster ID: d26f898925d741f9b5bf2b4f38e619f4
I20260709 13:02:51.510547  8595 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:02:51.554066  8595 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:02:51.555639  8595 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:02:51.572171  8595 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9: Generated new TSK 0
I20260709 13:02:51.572827  8595 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:02:51.635867  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:51.642149  8606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:51.642905  8607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:51.644272  8609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:51.645172  8021 server_base.cc:1061] running on GCE node
I20260709 13:02:51.645925  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:51.646099  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:51.646286  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602171646271 us; error 0 us; skew 500 ppm
I20260709 13:02:51.646843  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:51.649457  8021 webserver.cc:533] Webserver started at http://127.7.213.65:33717/ using document root <none> and password file <none>
I20260709 13:02:51.649946  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:51.650101  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:51.650418  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:51.651450  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-0-root/instance:
uuid: "a30440ab67344ff5bdb79368d2c87169"
format_stamp: "Formatted at 2026-07-09 13:02:51 on dist-test-slave-9g29"
I20260709 13:02:51.655966  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:02:51.659150  8614 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.659986  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:02:51.660287  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "a30440ab67344ff5bdb79368d2c87169"
format_stamp: "Formatted at 2026-07-09 13:02:51 on dist-test-slave-9g29"
I20260709 13:02:51.660552  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:51.675475  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:51.676795  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:51.678997  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:51.682006  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:51.682251  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.682549  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:51.682735  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.724532  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:41827
I20260709 13:02:51.724617  8676 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:41827 every 8 connection(s)
I20260709 13:02:51.729236  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:51.736863  8682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:51.738658  8681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:51.740525  8684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:51.740617  8021 server_base.cc:1061] running on GCE node
I20260709 13:02:51.741803  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:51.742005  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:51.742218  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602171742181 us; error 0 us; skew 500 ppm
I20260709 13:02:51.742739  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:51.745574  8021 webserver.cc:533] Webserver started at http://127.7.213.66:38913/ using document root <none> and password file <none>
I20260709 13:02:51.746163  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:51.746388  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:51.746680  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:51.747673  8677 heartbeater.cc:344] Connected to a master server at 127.7.213.126:40949
I20260709 13:02:51.748122  8677 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:51.748011  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-1-root/instance:
uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
format_stamp: "Formatted at 2026-07-09 13:02:51 on dist-test-slave-9g29"
I20260709 13:02:51.748872  8677 heartbeater.cc:507] Master 127.7.213.126:40949 requested a full tablet report, sending...
I20260709 13:02:51.751158  8550 ts_manager.cc:194] Registered new tserver with Master: a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827)
I20260709 13:02:51.753304  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:02:51.753425  8550 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39364
I20260709 13:02:51.757407  8689 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.758467  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:02:51.758728  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-1-root
uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
format_stamp: "Formatted at 2026-07-09 13:02:51 on dist-test-slave-9g29"
I20260709 13:02:51.758971  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:51.780809  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:51.781920  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:51.783347  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:51.785610  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:51.785786  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.786095  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:51.786237  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.826257  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.66:39009
I20260709 13:02:51.826365  8751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.66:39009 every 8 connection(s)
I20260709 13:02:51.830972  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:02:51.838191  8755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:02:51.838517  8756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:51.845371  8752 heartbeater.cc:344] Connected to a master server at 127.7.213.126:40949
I20260709 13:02:51.845789  8752 heartbeater.cc:461] Registering TS with master...
W20260709 13:02:51.846267  8758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:02:51.846725  8752 heartbeater.cc:507] Master 127.7.213.126:40949 requested a full tablet report, sending...
I20260709 13:02:51.847118  8021 server_base.cc:1061] running on GCE node
I20260709 13:02:51.847986  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:02:51.848253  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:02:51.848475  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602171848435 us; error 0 us; skew 500 ppm
I20260709 13:02:51.849010  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:02:51.849157  8550 ts_manager.cc:194] Registered new tserver with Master: bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:51.850899  8550 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39374
I20260709 13:02:51.851645  8021 webserver.cc:533] Webserver started at http://127.7.213.67:33427/ using document root <none> and password file <none>
I20260709 13:02:51.852258  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:02:51.852455  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:02:51.852743  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:02:51.853835  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-2-root/instance:
uuid: "7bafbb8448674ef6bad4f2343a88f517"
format_stamp: "Formatted at 2026-07-09 13:02:51 on dist-test-slave-9g29"
I20260709 13:02:51.858412  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:02:51.861984  8763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.862823  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:02:51.863193  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-2-root
uuid: "7bafbb8448674ef6bad4f2343a88f517"
format_stamp: "Formatted at 2026-07-09 13:02:51 on dist-test-slave-9g29"
I20260709 13:02:51.863487  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602158212224-8021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:02:51.875284  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:02:51.876590  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:02:51.878129  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:02:51.880767  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:02:51.880982  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.881299  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:02:51.881474  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:02:51.925750  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.67:42241
I20260709 13:02:51.925835  8825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.67:42241 every 8 connection(s)
I20260709 13:02:51.940593  8826 heartbeater.cc:344] Connected to a master server at 127.7.213.126:40949
I20260709 13:02:51.941197  8826 heartbeater.cc:461] Registering TS with master...
I20260709 13:02:51.941962  8826 heartbeater.cc:507] Master 127.7.213.126:40949 requested a full tablet report, sending...
I20260709 13:02:51.944069  8550 ts_manager.cc:194] Registered new tserver with Master: 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:51.944216  8021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.0151114s
I20260709 13:02:51.945454  8550 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39382
I20260709 13:02:51.970960  8550 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39398:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:02:51.973814  8550 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:02:52.058971  8789 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3ba99ef29b47d2b480955461e620f6 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:02:52.060984  8789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3ba99ef29b47d2b480955461e620f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.061192  8790 tablet_service.cc:1511] Processing CreateTablet for tablet db0a9299b66140fb91df4d8d4d329994 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:02:52.065182  8788 tablet_service.cc:1511] Processing CreateTablet for tablet a867b65c0da84df8b2f15ea4645de74b (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:02:52.059260  8791 tablet_service.cc:1511] Processing CreateTablet for tablet 3b944a2a26ec4833bdd2a5ae0c33ad94 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:02:52.070037  8787 tablet_service.cc:1511] Processing CreateTablet for tablet 62ada425b0dc451293343153da445ec1 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:02:52.070472  8786 tablet_service.cc:1511] Processing CreateTablet for tablet 7c20d8be02da4e3387df40c8466193dc (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:02:52.072163  8790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0a9299b66140fb91df4d8d4d329994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.072664  8642 tablet_service.cc:1511] Processing CreateTablet for tablet 3b944a2a26ec4833bdd2a5ae0c33ad94 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:02:52.074087  8642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b944a2a26ec4833bdd2a5ae0c33ad94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.076272  8786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c20d8be02da4e3387df40c8466193dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.077067  8641 tablet_service.cc:1511] Processing CreateTablet for tablet db0a9299b66140fb91df4d8d4d329994 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:02:52.078054  8791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b944a2a26ec4833bdd2a5ae0c33ad94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.078475  8641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0a9299b66140fb91df4d8d4d329994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.081195  8787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ada425b0dc451293343153da445ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.082717  8785 tablet_service.cc:1511] Processing CreateTablet for tablet 09fe7a618b30456080341949aafde463 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:02:52.085382  8640 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3ba99ef29b47d2b480955461e620f6 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:02:52.085942  8784 tablet_service.cc:1511] Processing CreateTablet for tablet 4a860cd902da4320bb1adc6e56613d0e (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:02:52.086841  8640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3ba99ef29b47d2b480955461e620f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.087391  8635 tablet_service.cc:1511] Processing CreateTablet for tablet 4a860cd902da4320bb1adc6e56613d0e (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:02:52.088264  8785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09fe7a618b30456080341949aafde463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.088898  8635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a860cd902da4320bb1adc6e56613d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.090098  8638 tablet_service.cc:1511] Processing CreateTablet for tablet 62ada425b0dc451293343153da445ec1 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:02:52.091439  8638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ada425b0dc451293343153da445ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.091478  8783 tablet_service.cc:1511] Processing CreateTablet for tablet a67e4eb9c5634896a1840bb1c1fb134d (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:02:52.092548  8784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a860cd902da4320bb1adc6e56613d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.088917  8639 tablet_service.cc:1511] Processing CreateTablet for tablet a867b65c0da84df8b2f15ea4645de74b (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:02:52.096935  8637 tablet_service.cc:1511] Processing CreateTablet for tablet 7c20d8be02da4e3387df40c8466193dc (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:02:52.097170  8782 tablet_service.cc:1511] Processing CreateTablet for tablet 17b52764e3634a389db130e9f210681c (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:02:52.097918  8783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67e4eb9c5634896a1840bb1c1fb134d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.098620  8782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b52764e3634a389db130e9f210681c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.099207  8633 tablet_service.cc:1511] Processing CreateTablet for tablet 17b52764e3634a389db130e9f210681c (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:02:52.099207  8636 tablet_service.cc:1511] Processing CreateTablet for tablet 09fe7a618b30456080341949aafde463 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:02:52.096845  8634 tablet_service.cc:1511] Processing CreateTablet for tablet a67e4eb9c5634896a1840bb1c1fb134d (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:02:52.097352  8639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a867b65c0da84df8b2f15ea4645de74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.104051  8636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09fe7a618b30456080341949aafde463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.104192  8788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a867b65c0da84df8b2f15ea4645de74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.105120  8633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b52764e3634a389db130e9f210681c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.108964  8634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67e4eb9c5634896a1840bb1c1fb134d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.112076  8637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c20d8be02da4e3387df40c8466193dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.148113  8714 tablet_service.cc:1511] Processing CreateTablet for tablet a867b65c0da84df8b2f15ea4645de74b (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:02:52.148938  8716 tablet_service.cc:1511] Processing CreateTablet for tablet db0a9299b66140fb91df4d8d4d329994 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:02:52.149674  8714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a867b65c0da84df8b2f15ea4645de74b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.150440  8716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db0a9299b66140fb91df4d8d4d329994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.151472  8712 tablet_service.cc:1511] Processing CreateTablet for tablet 7c20d8be02da4e3387df40c8466193dc (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:02:52.152909  8712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c20d8be02da4e3387df40c8466193dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.153015  8715 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3ba99ef29b47d2b480955461e620f6 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:02:52.154474  8715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3ba99ef29b47d2b480955461e620f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.147094  8717 tablet_service.cc:1511] Processing CreateTablet for tablet 3b944a2a26ec4833bdd2a5ae0c33ad94 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:02:52.157299  8710 tablet_service.cc:1511] Processing CreateTablet for tablet 4a860cd902da4320bb1adc6e56613d0e (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:02:52.158874  8710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a860cd902da4320bb1adc6e56613d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.161216  8709 tablet_service.cc:1511] Processing CreateTablet for tablet a67e4eb9c5634896a1840bb1c1fb134d (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:02:52.162190  8717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b944a2a26ec4833bdd2a5ae0c33ad94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.165274  8713 tablet_service.cc:1511] Processing CreateTablet for tablet 62ada425b0dc451293343153da445ec1 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:02:52.166751  8713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ada425b0dc451293343153da445ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.169040  8711 tablet_service.cc:1511] Processing CreateTablet for tablet 09fe7a618b30456080341949aafde463 (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:02:52.170351  8708 tablet_service.cc:1511] Processing CreateTablet for tablet 17b52764e3634a389db130e9f210681c (DEFAULT_TABLE table=test-workload [id=80948a33a39f43eeb9c64ec3f8f4379a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:02:52.176153  8711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09fe7a618b30456080341949aafde463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.180158  8708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b52764e3634a389db130e9f210681c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.185141  8709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67e4eb9c5634896a1840bb1c1fb134d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:52.212227  8841 tablet_bootstrap.cc:492] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.232985  8841 tablet_bootstrap.cc:654] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.245055  8841 tablet_bootstrap.cc:492] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.245636  8841 ts_tablet_manager.cc:1403] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.000s
I20260709 13:02:52.249555  8841 raft_consensus.cc:359] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.250469  8841 raft_consensus.cc:385] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.250885  8841 raft_consensus.cc:740] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.252282  8841 consensus_queue.cc:260] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.260892  8844 tablet_bootstrap.cc:492] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.262789  8841 ts_tablet_manager.cc:1434] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.011s
I20260709 13:02:52.263803  8826 heartbeater.cc:499] Master 127.7.213.126:40949 was elected leader, sending a full tablet report...
I20260709 13:02:52.267578  8841 tablet_bootstrap.cc:492] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.270692  8844 tablet_bootstrap.cc:654] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.274387  8841 tablet_bootstrap.cc:654] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.280190  8841 tablet_bootstrap.cc:492] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.280736  8841 ts_tablet_manager.cc:1403] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 13:02:52.282482  8844 tablet_bootstrap.cc:492] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.282944  8844 ts_tablet_manager.cc:1403] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.005s
I20260709 13:02:52.283599  8841 raft_consensus.cc:359] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.285900  8841 raft_consensus.cc:385] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.286196  8841 raft_consensus.cc:740] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.286427  8844 raft_consensus.cc:359] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.287137  8844 raft_consensus.cc:385] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.287055  8841 consensus_queue.cc:260] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.264786  8842 tablet_bootstrap.cc:492] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.289566  8841 ts_tablet_manager.cc:1434] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:02:52.287508  8844 raft_consensus.cc:740] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.290563  8841 tablet_bootstrap.cc:492] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.290766  8844 consensus_queue.cc:260] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.299038  8841 tablet_bootstrap.cc:654] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.300247  8842 tablet_bootstrap.cc:654] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.304034  8844 ts_tablet_manager.cc:1434] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.021s	user 0.011s	sys 0.000s
I20260709 13:02:52.304785  8752 heartbeater.cc:499] Master 127.7.213.126:40949 was elected leader, sending a full tablet report...
I20260709 13:02:52.309865  8844 tablet_bootstrap.cc:492] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.316231  8844 tablet_bootstrap.cc:654] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.317451  8848 raft_consensus.cc:493] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.318130  8842 tablet_bootstrap.cc:492] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.317930  8848 raft_consensus.cc:515] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.318588  8842 ts_tablet_manager.cc:1403] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.054s	user 0.006s	sys 0.008s
I20260709 13:02:52.323083  8842 raft_consensus.cc:359] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.323966  8842 raft_consensus.cc:385] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.324342  8842 raft_consensus.cc:740] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.325812  8848 leader_election.cc:290] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241), a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827)
I20260709 13:02:52.328903  8841 tablet_bootstrap.cc:492] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.329409  8841 ts_tablet_manager.cc:1403] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.016s
I20260709 13:02:52.333158  8841 raft_consensus.cc:359] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.333900  8841 raft_consensus.cc:385] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.333671  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
I20260709 13:02:52.334244  8841 raft_consensus.cc:740] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.335005  8841 consensus_queue.cc:260] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
W20260709 13:02:52.335353  8692 leader_election.cc:343] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.337281  8841 ts_tablet_manager.cc:1434] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:02:52.338266  8841 tablet_bootstrap.cc:492] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.338450  8844 tablet_bootstrap.cc:492] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.338876  8844 ts_tablet_manager.cc:1403] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.029s	user 0.000s	sys 0.009s
I20260709 13:02:52.341754  8844 raft_consensus.cc:359] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.342542  8844 raft_consensus.cc:385] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.342854  8844 raft_consensus.cc:740] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.344112  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.344074  8842 consensus_queue.cc:260] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.344750  8841 tablet_bootstrap.cc:654] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:52.345886  8691 leader_election.cc:343] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.343588  8844 consensus_queue.cc:260] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.346355  8691 leader_election.cc:304] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb841c52fb5b41288d1dd2a9967e07b3; no voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:52.347559  8848 raft_consensus.cc:2749] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:52.348205  8845 raft_consensus.cc:493] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.348645  8844 ts_tablet_manager.cc:1434] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:02:52.348663  8845 raft_consensus.cc:515] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.352746  8845 leader_election.cc:290] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.353186  8844 tablet_bootstrap.cc:492] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.360581  8844 tablet_bootstrap.cc:654] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.367944  8845 raft_consensus.cc:493] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.368626  8845 raft_consensus.cc:515] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.372985  8845 leader_election.cc:290] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.376030  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.377831  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3ba99ef29b47d2b480955461e620f6" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
W20260709 13:02:52.380144  8765 leader_election.cc:343] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:52.381206  8765 leader_election.cc:343] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.382715  8848 raft_consensus.cc:493] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.383287  8848 raft_consensus.cc:515] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.385453  8848 leader_election.cc:290] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.388044  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
I20260709 13:02:52.389640  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
W20260709 13:02:52.391477  8691 leader_election.cc:343] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:02:52.396366  8692 leader_election.cc:343] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.396988  8692 leader_election.cc:304] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb841c52fb5b41288d1dd2a9967e07b3; no voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:52.398217  8848 raft_consensus.cc:2749] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:52.400228  8844 tablet_bootstrap.cc:492] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.400938  8844 ts_tablet_manager.cc:1403] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.048s	user 0.017s	sys 0.022s
I20260709 13:02:52.401202  8853 raft_consensus.cc:493] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.401690  8853 raft_consensus.cc:515] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.401965  8842 ts_tablet_manager.cc:1434] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.083s	user 0.025s	sys 0.035s
I20260709 13:02:52.403965  8677 heartbeater.cc:499] Master 127.7.213.126:40949 was elected leader, sending a full tablet report...
I20260709 13:02:52.405018  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
I20260709 13:02:52.406502  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3ba99ef29b47d2b480955461e620f6" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
I20260709 13:02:52.403898  8844 raft_consensus.cc:359] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.407532  8844 raft_consensus.cc:385] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:02:52.407734  8764 leader_election.cc:343] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.408416  8842 tablet_bootstrap.cc:492] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.408459  8764 leader_election.cc:304] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517; no voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3
I20260709 13:02:52.409943  8845 raft_consensus.cc:2749] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:52.407876  8844 raft_consensus.cc:740] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.411643  8853 leader_election.cc:290] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.411209  8844 consensus_queue.cc:260] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.412457  8726 raft_consensus.cc:2468] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 0.
I20260709 13:02:52.416136  8842 tablet_bootstrap.cc:654] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.416313  8764 leader_election.cc:304] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:52.417519  8844 ts_tablet_manager.cc:1434] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:02:52.417464  8845 raft_consensus.cc:2804] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:52.418123  8845 raft_consensus.cc:493] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.418551  8845 raft_consensus.cc:3060] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.427207  8844 tablet_bootstrap.cc:492] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.429911  8845 raft_consensus.cc:515] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.437420  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
W20260709 13:02:52.439164  8617 leader_election.cc:343] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.439827  8841 tablet_bootstrap.cc:492] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.440145  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3ba99ef29b47d2b480955461e620f6" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:52.440501  8841 ts_tablet_manager.cc:1403] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.102s	user 0.017s	sys 0.019s
I20260709 13:02:52.440900  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3ba99ef29b47d2b480955461e620f6" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:52.441478  8726 raft_consensus.cc:3060] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:02:52.441572  8765 leader_election.cc:343] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.442485  8844 tablet_bootstrap.cc:654] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.444507  8841 raft_consensus.cc:359] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.450449  8845 leader_election.cc:290] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.450909  8841 raft_consensus.cc:385] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.451327  8726 raft_consensus.cc:2468] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 1.
I20260709 13:02:52.451439  8841 raft_consensus.cc:740] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.452842  8764 leader_election.cc:304] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:52.452639  8841 consensus_queue.cc:260] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.454030  8845 raft_consensus.cc:2804] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:52.454980  8844 tablet_bootstrap.cc:492] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.455622  8844 ts_tablet_manager.cc:1403] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.003s
I20260709 13:02:52.459523  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
W20260709 13:02:52.461741  8615 leader_election.cc:343] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.462352  8615 leader_election.cc:304] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169; no voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3
I20260709 13:02:52.463474  8841 ts_tablet_manager.cc:1434] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.005s
I20260709 13:02:52.463413  8853 raft_consensus.cc:2749] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:52.464605  8841 tablet_bootstrap.cc:492] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.466228  8844 raft_consensus.cc:359] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.467450  8844 raft_consensus.cc:385] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.467967  8844 raft_consensus.cc:740] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.469024  8844 consensus_queue.cc:260] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.471374  8841 tablet_bootstrap.cc:654] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.473309  8844 ts_tablet_manager.cc:1434] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:02:52.474735  8844 tablet_bootstrap.cc:492] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.478367  8842 tablet_bootstrap.cc:492] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.479106  8842 ts_tablet_manager.cc:1403] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.071s	user 0.030s	sys 0.018s
I20260709 13:02:52.482260  8844 tablet_bootstrap.cc:654] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.483388  8842 raft_consensus.cc:359] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.484666  8842 raft_consensus.cc:385] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.485109  8842 raft_consensus.cc:740] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.485198  8845 raft_consensus.cc:697] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:52.486440  8845 consensus_queue.cc:237] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.486541  8842 consensus_queue.cc:260] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.489979  8842 ts_tablet_manager.cc:1434] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.004s
I20260709 13:02:52.491427  8842 tablet_bootstrap.cc:492] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.494091  8844 tablet_bootstrap.cc:492] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.494611  8844 ts_tablet_manager.cc:1403] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.005s
I20260709 13:02:52.497617  8844 raft_consensus.cc:359] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.498561  8844 raft_consensus.cc:385] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.498934  8844 raft_consensus.cc:740] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.499935  8844 consensus_queue.cc:260] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.501562  8842 tablet_bootstrap.cc:654] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
W20260709 13:02:52.502390  8598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:02:52.504516  8848 raft_consensus.cc:493] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.504989  8848 raft_consensus.cc:515] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.507282  8848 leader_election.cc:290] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.509814  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.511231  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
W20260709 13:02:52.512465  8691 leader_election.cc:343] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.513381  8844 ts_tablet_manager.cc:1434] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
W20260709 13:02:52.513589  8692 leader_election.cc:343] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.513990  8692 leader_election.cc:304] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb841c52fb5b41288d1dd2a9967e07b3; no voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:52.514711  8841 tablet_bootstrap.cc:492] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.515113  8848 raft_consensus.cc:2749] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:52.515417  8841 ts_tablet_manager.cc:1403] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.051s	user 0.022s	sys 0.024s
I20260709 13:02:52.516917  8844 tablet_bootstrap.cc:492] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.521632  8841 raft_consensus.cc:359] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.522583  8841 raft_consensus.cc:385] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.522945  8841 raft_consensus.cc:740] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.523366  8842 tablet_bootstrap.cc:492] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.523905  8842 ts_tablet_manager.cc:1403] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.000s
I20260709 13:02:52.525761  8844 tablet_bootstrap.cc:654] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.523918  8841 consensus_queue.cc:260] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.526844  8842 raft_consensus.cc:359] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.527597  8842 raft_consensus.cc:385] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.527904  8842 raft_consensus.cc:740] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.529019  8841 ts_tablet_manager.cc:1434] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:02:52.528647  8842 consensus_queue.cc:260] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.530079  8841 tablet_bootstrap.cc:492] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.531140  8842 ts_tablet_manager.cc:1434] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:02:52.530527  8549 catalog_manager.cc:5697] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:52.532323  8842 tablet_bootstrap.cc:492] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.538228  8841 tablet_bootstrap.cc:654] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.539150  8842 tablet_bootstrap.cc:654] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.556100  8844 tablet_bootstrap.cc:492] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.556998  8844 ts_tablet_manager.cc:1403] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.014s
I20260709 13:02:52.558247  8848 raft_consensus.cc:493] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.558755  8848 raft_consensus.cc:515] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.561017  8848 leader_election.cc:290] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.562134  8844 raft_consensus.cc:359] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.562950  8844 raft_consensus.cc:385] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.562912  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b52764e3634a389db130e9f210681c" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.563396  8844 raft_consensus.cc:740] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.564872  8844 consensus_queue.cc:260] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
W20260709 13:02:52.565639  8691 leader_election.cc:343] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.566020  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b52764e3634a389db130e9f210681c" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
I20260709 13:02:52.566622  8801 raft_consensus.cc:2468] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 0.
I20260709 13:02:52.567133  8841 tablet_bootstrap.cc:492] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.567615  8841 ts_tablet_manager.cc:1403] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.038s	user 0.030s	sys 0.003s
I20260709 13:02:52.567891  8692 leader_election.cc:304] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:52.568207  8844 ts_tablet_manager.cc:1434] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:02:52.569087  8848 raft_consensus.cc:2804] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:52.569551  8848 raft_consensus.cc:493] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.569988  8848 raft_consensus.cc:3060] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.570515  8841 raft_consensus.cc:359] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.571090  8842 tablet_bootstrap.cc:492] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.571264  8841 raft_consensus.cc:385] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.571548  8842 ts_tablet_manager.cc:1403] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.010s
I20260709 13:02:52.571563  8841 raft_consensus.cc:740] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.572490  8841 consensus_queue.cc:260] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.569228  8844 tablet_bootstrap.cc:492] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.574904  8841 ts_tablet_manager.cc:1434] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:02:52.575884  8841 tablet_bootstrap.cc:492] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.577731  8848 raft_consensus.cc:515] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.574406  8842 raft_consensus.cc:359] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.580693  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b52764e3634a389db130e9f210681c" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:52.582007  8845 raft_consensus.cc:493] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.582505  8841 tablet_bootstrap.cc:654] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.582482  8845 raft_consensus.cc:515] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.584673  8844 tablet_bootstrap.cc:654] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.584734  8845 leader_election.cc:290] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
W20260709 13:02:52.586810  8691 leader_election.cc:343] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.580585  8842 raft_consensus.cc:385] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.588446  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.588666  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
I20260709 13:02:52.588696  8842 raft_consensus.cc:740] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.589123  8651 raft_consensus.cc:2468] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 0.
I20260709 13:02:52.589915  8842 consensus_queue.cc:260] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.590502  8765 leader_election.cc:304] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
W20260709 13:02:52.591553  8764 leader_election.cc:343] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.592486  8845 raft_consensus.cc:2804] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:52.592919  8845 raft_consensus.cc:493] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.593326  8845 raft_consensus.cc:3060] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.600085  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b52764e3634a389db130e9f210681c" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:52.592928  8842 ts_tablet_manager.cc:1434] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:02:52.601863  8842 tablet_bootstrap.cc:492] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.600771  8801 raft_consensus.cc:3060] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.610186  8842 tablet_bootstrap.cc:654] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.603015  8845 raft_consensus.cc:515] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.612613  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:52.613179  8651 raft_consensus.cc:3060] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.620043  8845 leader_election.cc:290] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.621577  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:52.622828  8801 raft_consensus.cc:2468] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 1.
W20260709 13:02:52.623070  8764 leader_election.cc:343] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:02:52.624028  8692 leader_election.cc:304] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:52.624184  8651 raft_consensus.cc:2468] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 1.
I20260709 13:02:52.625571  8765 leader_election.cc:304] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: bb841c52fb5b41288d1dd2a9967e07b3
I20260709 13:02:52.627031  8845 raft_consensus.cc:2804] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:52.627941  8845 raft_consensus.cc:697] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:52.628935  8845 consensus_queue.cc:237] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.600723  8848 leader_election.cc:290] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.631287  8848 raft_consensus.cc:2804] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:52.632900  8848 raft_consensus.cc:697] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Becoming Leader. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:52.633713  8848 consensus_queue.cc:237] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.637739  8841 tablet_bootstrap.cc:492] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.638430  8841 ts_tablet_manager.cc:1403] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.063s	user 0.027s	sys 0.033s
I20260709 13:02:52.642355  8841 raft_consensus.cc:359] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.643421  8841 raft_consensus.cc:385] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.643826  8841 raft_consensus.cc:740] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.644961  8841 consensus_queue.cc:260] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.647966  8841 ts_tablet_manager.cc:1434] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.006s
I20260709 13:02:52.649502  8841 tablet_bootstrap.cc:492] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.650107  8548 catalog_manager.cc:5697] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:52.658242  8841 tablet_bootstrap.cc:654] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.659543  8844 tablet_bootstrap.cc:492] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.660094  8844 ts_tablet_manager.cc:1403] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.091s	user 0.040s	sys 0.036s
I20260709 13:02:52.660297  8842 tablet_bootstrap.cc:492] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.660780  8842 ts_tablet_manager.cc:1403] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.059s	user 0.020s	sys 0.009s
I20260709 13:02:52.662751  8844 raft_consensus.cc:359] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.663350  8844 raft_consensus.cc:385] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.663602  8844 raft_consensus.cc:740] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.664625  8844 consensus_queue.cc:260] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.666746  8844 ts_tablet_manager.cc:1434] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:02:52.667613  8844 tablet_bootstrap.cc:492] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.668260  8842 raft_consensus.cc:359] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.668807  8842 raft_consensus.cc:385] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.669096  8842 raft_consensus.cc:740] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.669945  8842 consensus_queue.cc:260] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.673384  8842 ts_tablet_manager.cc:1434] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:02:52.674067  8842 tablet_bootstrap.cc:492] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.674400  8844 tablet_bootstrap.cc:654] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.677644  8841 tablet_bootstrap.cc:492] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.678124  8841 ts_tablet_manager.cc:1403] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.004s
I20260709 13:02:52.680914  8841 raft_consensus.cc:359] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.681731  8841 raft_consensus.cc:385] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.682055  8841 raft_consensus.cc:740] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.682595  8842 tablet_bootstrap.cc:654] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.683012  8841 consensus_queue.cc:260] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.684986  8841 ts_tablet_manager.cc:1434] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:02:52.687001  8841 tablet_bootstrap.cc:492] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.690440  8549 catalog_manager.cc:5697] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:52.694371  8841 tablet_bootstrap.cc:654] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.707307  8841 tablet_bootstrap.cc:492] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.707962  8841 ts_tablet_manager.cc:1403] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.011s
I20260709 13:02:52.711597  8841 raft_consensus.cc:359] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.712456  8841 raft_consensus.cc:385] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.712786  8841 raft_consensus.cc:740] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.713809  8841 consensus_queue.cc:260] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.719213  8844 tablet_bootstrap.cc:492] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.719614  8841 ts_tablet_manager.cc:1434] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:02:52.719702  8844 ts_tablet_manager.cc:1403] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.052s	user 0.026s	sys 0.023s
I20260709 13:02:52.720727  8841 tablet_bootstrap.cc:492] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517: Bootstrap starting.
I20260709 13:02:52.722083  8842 tablet_bootstrap.cc:492] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.722618  8842 ts_tablet_manager.cc:1403] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.027s
I20260709 13:02:52.722501  8844 raft_consensus.cc:359] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.723222  8844 raft_consensus.cc:385] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.723497  8844 raft_consensus.cc:740] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.724148  8844 consensus_queue.cc:260] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.725883  8842 raft_consensus.cc:359] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.726387  8844 ts_tablet_manager.cc:1434] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:02:52.726635  8841 tablet_bootstrap.cc:654] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.726675  8842 raft_consensus.cc:385] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.727159  8842 raft_consensus.cc:740] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.727289  8844 tablet_bootstrap.cc:492] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.728236  8842 consensus_queue.cc:260] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.735342  8844 tablet_bootstrap.cc:654] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.740444  8842 ts_tablet_manager.cc:1434] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:02:52.745052  8853 raft_consensus.cc:493] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.745695  8853 raft_consensus.cc:515] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.748508  8853 leader_election.cc:290] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.750497  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
I20260709 13:02:52.751359  8801 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 0.
I20260709 13:02:52.751483  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
I20260709 13:02:52.752257  8726 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 0.
I20260709 13:02:52.752733  8844 tablet_bootstrap.cc:492] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.753216  8844 ts_tablet_manager.cc:1403] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.007s
I20260709 13:02:52.753363  8617 leader_election.cc:304] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:52.754719  8857 raft_consensus.cc:2804] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:52.755178  8857 raft_consensus.cc:493] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.755177  8844 raft_consensus.cc:359] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.755761  8857 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.755995  8844 raft_consensus.cc:385] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.756291  8844 raft_consensus.cc:740] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.757081  8844 consensus_queue.cc:260] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.761963  8844 ts_tablet_manager.cc:1434] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:02:52.762907  8844 tablet_bootstrap.cc:492] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3: Bootstrap starting.
I20260709 13:02:52.762995  8842 tablet_bootstrap.cc:492] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.769142  8844 tablet_bootstrap.cc:654] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.771399  8857 raft_consensus.cc:515] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.775981  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:52.776871  8801 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.778595  8842 tablet_bootstrap.cc:654] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.778656  8841 tablet_bootstrap.cc:492] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517: No bootstrap required, opened a new log
I20260709 13:02:52.779289  8841 ts_tablet_manager.cc:1403] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.044s
I20260709 13:02:52.782367  8841 raft_consensus.cc:359] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.783382  8841 raft_consensus.cc:385] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.783841  8841 raft_consensus.cc:740] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.784906  8841 consensus_queue.cc:260] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.787655  8841 ts_tablet_manager.cc:1434] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:02:52.791280  8857 leader_election.cc:290] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 election: Requested vote from peers 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.792302  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:52.792999  8801 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 1.
I20260709 13:02:52.793195  8726 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.796932  8844 tablet_bootstrap.cc:492] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3: No bootstrap required, opened a new log
I20260709 13:02:52.796753  8617 leader_election.cc:304] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:52.797430  8844 ts_tablet_manager.cc:1403] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.019s
I20260709 13:02:52.797823  8857 raft_consensus.cc:2804] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:52.798501  8842 tablet_bootstrap.cc:492] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.799054  8842 ts_tablet_manager.cc:1403] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.010s
I20260709 13:02:52.800509  8844 raft_consensus.cc:359] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.801251  8844 raft_consensus.cc:385] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.801551  8844 raft_consensus.cc:740] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.802187  8842 raft_consensus.cc:359] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.802414  8844 consensus_queue.cc:260] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.802949  8842 raft_consensus.cc:385] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.803095  8726 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 1.
I20260709 13:02:52.798408  8857 raft_consensus.cc:697] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 1 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:52.803373  8842 raft_consensus.cc:740] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.804291  8857 consensus_queue.cc:237] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:52.804446  8842 consensus_queue.cc:260] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.805968  8844 ts_tablet_manager.cc:1434] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:02:52.806736  8842 ts_tablet_manager.cc:1434] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.006s
I20260709 13:02:52.807735  8842 tablet_bootstrap.cc:492] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.814500  8842 tablet_bootstrap.cc:654] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.814535  8548 catalog_manager.cc:5697] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 0 to 1, leader changed from <none> to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:52.821841  8842 tablet_bootstrap.cc:492] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.822249  8842 ts_tablet_manager.cc:1403] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.007s
I20260709 13:02:52.824433  8848 raft_consensus.cc:493] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.824911  8848 raft_consensus.cc:515] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.825274  8842 raft_consensus.cc:359] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.825919  8842 raft_consensus.cc:385] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.826179  8842 raft_consensus.cc:740] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.826970  8848 leader_election.cc:290] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.826843  8842 consensus_queue.cc:260] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.827797  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.828488  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
I20260709 13:02:52.829092  8801 raft_consensus.cc:2468] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 0.
I20260709 13:02:52.829396  8842 ts_tablet_manager.cc:1434] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:02:52.828462  8651 raft_consensus.cc:2468] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 0.
I20260709 13:02:52.830271  8842 tablet_bootstrap.cc:492] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.830201  8692 leader_election.cc:304] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:52.831190  8848 raft_consensus.cc:2804] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:52.831517  8848 raft_consensus.cc:493] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.831869  8848 raft_consensus.cc:3060] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.836519  8842 tablet_bootstrap.cc:654] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.839020  8848 raft_consensus.cc:515] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.842813  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:52.843480  8651 raft_consensus.cc:3060] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.843807  8848 leader_election.cc:290] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.845057  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:52.845641  8842 tablet_bootstrap.cc:492] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.845664  8801 raft_consensus.cc:3060] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.846262  8842 ts_tablet_manager.cc:1403] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 13:02:52.848532  8651 raft_consensus.cc:2468] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 1.
I20260709 13:02:52.849120  8842 raft_consensus.cc:359] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.849592  8691 leader_election.cc:304] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:52.849925  8842 raft_consensus.cc:385] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.850265  8842 raft_consensus.cc:740] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.850443  8848 raft_consensus.cc:2804] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:52.851025  8848 raft_consensus.cc:697] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Becoming Leader. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:52.850996  8842 consensus_queue.cc:260] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.852053  8848 consensus_queue.cc:237] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.853452  8842 ts_tablet_manager.cc:1434] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:02:52.853590  8801 raft_consensus.cc:2468] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 1.
I20260709 13:02:52.854614  8842 tablet_bootstrap.cc:492] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169: Bootstrap starting.
I20260709 13:02:52.858810  8848 raft_consensus.cc:493] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.859290  8848 raft_consensus.cc:515] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.861315  8842 tablet_bootstrap.cc:654] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169: Neither blocks nor log segments found. Creating new log.
I20260709 13:02:52.861330  8550 catalog_manager.cc:5697] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:52.862804  8876 raft_consensus.cc:493] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.862363  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.864321  8651 raft_consensus.cc:2393] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bb841c52fb5b41288d1dd2a9967e07b3 in current term 1: Already voted for candidate 7bafbb8448674ef6bad4f2343a88f517 in this term.
I20260709 13:02:52.869544  8848 leader_election.cc:290] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.872004  8845 raft_consensus.cc:493] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.876749  8845 raft_consensus.cc:515] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.878856  8845 leader_election.cc:290] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.874878  8863 raft_consensus.cc:493] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.879935  8863 raft_consensus.cc:515] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.880399  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
I20260709 13:02:52.863335  8876 raft_consensus.cc:515] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.880843  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.881029  8726 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 0.
I20260709 13:02:52.881434  8652 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 0.
I20260709 13:02:52.882057  8863 leader_election.cc:290] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.882324  8876 leader_election.cc:290] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.877018  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
I20260709 13:02:52.882238  8764 leader_election.cc:304] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:52.883278  8842 tablet_bootstrap.cc:492] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169: No bootstrap required, opened a new log
I20260709 13:02:52.883895  8863 raft_consensus.cc:2804] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:52.884220  8863 raft_consensus.cc:493] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.884310  8842 ts_tablet_manager.cc:1403] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.005s
I20260709 13:02:52.884572  8863 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.884768  8692 leader_election.cc:304] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb841c52fb5b41288d1dd2a9967e07b3; no voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:52.885042  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67e4eb9c5634896a1840bb1c1fb134d" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
I20260709 13:02:52.885648  8726 raft_consensus.cc:2468] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 0.
I20260709 13:02:52.886134  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
I20260709 13:02:52.886722  8801 raft_consensus.cc:2468] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 0.
I20260709 13:02:52.886804  8764 leader_election.cc:304] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:52.887279  8842 raft_consensus.cc:359] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.887611  8876 raft_consensus.cc:3060] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.887617  8845 raft_consensus.cc:2804] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:52.887986  8842 raft_consensus.cc:385] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:02:52.888231  8845 raft_consensus.cc:493] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.888332  8842 raft_consensus.cc:740] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Initialized, Role: FOLLOWER
I20260709 13:02:52.888676  8845 raft_consensus.cc:3060] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.889072  8842 consensus_queue.cc:260] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.890087  8692 leader_election.cc:304] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:52.891029  8848 raft_consensus.cc:2804] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:52.891379  8848 raft_consensus.cc:493] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.891458  8842 ts_tablet_manager.cc:1434] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:02:52.891795  8848 raft_consensus.cc:3060] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.891520  8863 raft_consensus.cc:515] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.892360  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67e4eb9c5634896a1840bb1c1fb134d" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.892963  8651 raft_consensus.cc:2468] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 0.
I20260709 13:02:52.894685  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:52.895275  8652 raft_consensus.cc:2468] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 0.
I20260709 13:02:52.896117  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:52.896643  8651 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.895519  8863 leader_election.cc:290] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.900753  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:52.901520  8726 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.902457  8845 raft_consensus.cc:515] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.905709  8651 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 1.
I20260709 13:02:52.906888  8652 raft_consensus.cc:3060] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.908258  8876 raft_consensus.cc:2749] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:52.908737  8765 leader_election.cc:304] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:52.909073  8726 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 1.
I20260709 13:02:52.909571  8845 leader_election.cc:290] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:52.910197  8845 raft_consensus.cc:2804] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:52.910676  8845 raft_consensus.cc:697] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:52.910722  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67e4eb9c5634896a1840bb1c1fb134d" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:52.911412  8727 raft_consensus.cc:3060] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.911664  8845 consensus_queue.cc:237] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.911878  8848 raft_consensus.cc:515] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.914005  8857 raft_consensus.cc:493] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.914518  8857 raft_consensus.cc:515] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.915413  8848 leader_election.cc:290] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.916527  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:52.916446  8650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:52.917280  8801 raft_consensus.cc:3060] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.917488  8650 raft_consensus.cc:3060] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.919358  8727 raft_consensus.cc:2468] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 1.
I20260709 13:02:52.920686  8764 leader_election.cc:304] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:52.921762  8862 raft_consensus.cc:2804] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:52.922240  8862 raft_consensus.cc:697] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:52.923079  8862 consensus_queue.cc:237] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.924028  8857 leader_election.cc:290] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:52.925495  8801 raft_consensus.cc:2468] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 1.
I20260709 13:02:52.926609  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b52764e3634a389db130e9f210681c" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
I20260709 13:02:52.927476  8692 leader_election.cc:304] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:52.926682  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b52764e3634a389db130e9f210681c" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
I20260709 13:02:52.928853  8848 raft_consensus.cc:2804] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:52.929337  8800 raft_consensus.cc:2393] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a30440ab67344ff5bdb79368d2c87169 in current term 1: Already voted for candidate bb841c52fb5b41288d1dd2a9967e07b3 in this term.
I20260709 13:02:52.929687  8848 raft_consensus.cc:697] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Becoming Leader. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:52.930624  8848 consensus_queue.cc:237] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:52.931106  8615 leader_election.cc:304] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169; no voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3
I20260709 13:02:52.937207  8857 raft_consensus.cc:3060] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.942487  8550 catalog_manager.cc:5697] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:52.948701  8650 raft_consensus.cc:2468] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 1.
I20260709 13:02:52.950379  8548 catalog_manager.cc:5697] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:52.950385  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67e4eb9c5634896a1840bb1c1fb134d" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:52.953210  8651 raft_consensus.cc:3060] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:52.952939  8845 consensus_queue.cc:1048] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:52.954145  8857 raft_consensus.cc:2749] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:02:52.959285  8549 catalog_manager.cc:5697] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:52.974468  8651 raft_consensus.cc:2468] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 1.
I20260709 13:02:52.984462  8863 consensus_queue.cc:1048] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:52.999301  8857 raft_consensus.cc:493] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.999804  8857 raft_consensus.cc:515] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.001889  8857 leader_election.cc:290] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:53.002529  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ada425b0dc451293343153da445ec1" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
I20260709 13:02:53.003237  8726 raft_consensus.cc:2468] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 0.
I20260709 13:02:53.003265  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ada425b0dc451293343153da445ec1" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517" is_pre_election: true
I20260709 13:02:53.003849  8800 raft_consensus.cc:2468] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 0.
I20260709 13:02:53.004242  8615 leader_election.cc:304] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:53.005046  8857 raft_consensus.cc:2804] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:53.005467  8857 raft_consensus.cc:493] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.005744  8857 raft_consensus.cc:3060] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:53.010354  8857 raft_consensus.cc:515] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.012465  8857 leader_election.cc:290] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 election: Requested vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:53.013423  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ada425b0dc451293343153da445ec1" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:53.013726  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ada425b0dc451293343153da445ec1" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:53.014115  8726 raft_consensus.cc:3060] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:53.014259  8800 raft_consensus.cc:3060] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:53.021255  8726 raft_consensus.cc:2468] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 1.
I20260709 13:02:53.021255  8800 raft_consensus.cc:2468] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 1.
I20260709 13:02:53.022774  8615 leader_election.cc:304] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:53.023494  8857 raft_consensus.cc:2804] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:53.024042  8857 raft_consensus.cc:697] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 1 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:53.024757  8857 consensus_queue.cc:237] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.032411  8550 catalog_manager.cc:5697] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 0 to 1, leader changed from <none> to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:53.066326  8863 raft_consensus.cc:493] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.066939  8863 raft_consensus.cc:515] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.068931  8863 leader_election.cc:290] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:53.069883  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169" is_pre_election: true
I20260709 13:02:53.070122  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" is_pre_election: true
I20260709 13:02:53.070644  8652 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 0.
I20260709 13:02:53.070773  8726 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 0.
I20260709 13:02:53.072001  8765 leader_election.cc:304] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:53.072722  8863 raft_consensus.cc:2804] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:02:53.073060  8863 raft_consensus.cc:493] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.073437  8863 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:53.078457  8863 raft_consensus.cc:515] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.080608  8863 leader_election.cc:290] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:53.081651  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:53.082000  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:53.082207  8652 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:53.082628  8726 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:02:53.087256  8652 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 1.
I20260709 13:02:53.088294  8726 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 1.
I20260709 13:02:53.088578  8765 leader_election.cc:304] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:53.089573  8863 raft_consensus.cc:2804] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:02:53.090109  8863 raft_consensus.cc:697] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:53.091001  8863 consensus_queue.cc:237] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.098951  8550 catalog_manager.cc:5697] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:53.185941  8848 consensus_queue.cc:1048] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.193323  8863 consensus_queue.cc:1048] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:53.199525  8848 consensus_queue.cc:1048] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.208973  8862 consensus_queue.cc:1048] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:53.246701  8726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
mode: GRACEFUL
new_leader_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:56246
I20260709 13:02:53.247299  8726 raft_consensus.cc:606] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Received request to transfer leadership to TS a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:53.253911  8857 consensus_queue.cc:1048] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.263921  8801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f3ba99ef29b47d2b480955461e620f6"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
mode: GRACEFUL
new_leader_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:39630
I20260709 13:02:53.264536  8801 raft_consensus.cc:606] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Received request to transfer leadership to TS a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:53.268874  8853 consensus_queue.cc:1048] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.271682  8801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
mode: GRACEFUL
new_leader_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:39630
I20260709 13:02:53.272298  8801 raft_consensus.cc:606] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Received request to transfer leadership to TS a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:53.282797  8801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
mode: GRACEFUL
new_leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:39630
I20260709 13:02:53.283389  8801 raft_consensus.cc:606] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Received request to transfer leadership to TS bb841c52fb5b41288d1dd2a9967e07b3
I20260709 13:02:53.288578  8801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09fe7a618b30456080341949aafde463"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
mode: GRACEFUL
new_leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:39630
I20260709 13:02:53.289130  8801 raft_consensus.cc:606] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Received request to transfer leadership to TS bb841c52fb5b41288d1dd2a9967e07b3
I20260709 13:02:53.296731  8801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a67e4eb9c5634896a1840bb1c1fb134d"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
mode: GRACEFUL
new_leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:39630
I20260709 13:02:53.297499  8801 raft_consensus.cc:606] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Received request to transfer leadership to TS bb841c52fb5b41288d1dd2a9967e07b3
I20260709 13:02:53.303511  8726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17b52764e3634a389db130e9f210681c"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
mode: GRACEFUL
new_leader_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:56246
I20260709 13:02:53.304207  8726 raft_consensus.cc:606] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Received request to transfer leadership to TS 7bafbb8448674ef6bad4f2343a88f517
I20260709 13:02:53.324183  8848 consensus_queue.cc:1048] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.329306  8845 consensus_queue.cc:1048] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.337472  8876 consensus_queue.cc:1048] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.343931  8863 consensus_queue.cc:1048] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.355050  8862 raft_consensus.cc:993] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517: : Instructing follower bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:02:53.355531  8862 raft_consensus.cc:1081] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Signalling peer bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:02:53.357872  8726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a67e4eb9c5634896a1840bb1c1fb134d"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:02:53.358413  8726 raft_consensus.cc:493] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:53.358824  8726 raft_consensus.cc:3060] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.363660  8726 raft_consensus.cc:515] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.365860  8726 leader_election.cc:290] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 2 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:53.366549  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67e4eb9c5634896a1840bb1c1fb134d" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:53.366855  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a67e4eb9c5634896a1840bb1c1fb134d" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:53.367177  8652 raft_consensus.cc:3060] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.367389  8801 raft_consensus.cc:3055] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:53.367765  8801 raft_consensus.cc:740] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:53.368649  8801 consensus_queue.cc:260] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.369843  8801 raft_consensus.cc:3060] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.373972  8652 raft_consensus.cc:2468] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 2.
I20260709 13:02:53.375291  8691 leader_election.cc:304] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:53.376029  8801 raft_consensus.cc:2468] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 2.
I20260709 13:02:53.376118  8876 raft_consensus.cc:2804] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:53.376672  8876 raft_consensus.cc:697] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Becoming Leader. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:53.377583  8876 consensus_queue.cc:237] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.384467  8547 catalog_manager.cc:5697] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 reported cstate change: term changed from 1 to 2, leader changed from 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67) to bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:53.402511  8845 consensus_queue.cc:1048] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.404560  8863 raft_consensus.cc:993] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517: : Instructing follower a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:53.404964  8863 raft_consensus.cc:1081] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Signalling peer a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:53.406436  8651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f3ba99ef29b47d2b480955461e620f6"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:40444
I20260709 13:02:53.406942  8651 raft_consensus.cc:493] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:53.407383  8651 raft_consensus.cc:3060] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.410538  8862 raft_consensus.cc:993] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517: : Instructing follower a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:53.411054  8862 raft_consensus.cc:1081] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Signalling peer a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:53.414942  8651 raft_consensus.cc:515] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.415104  8863 consensus_queue.cc:1048] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.417852  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3ba99ef29b47d2b480955461e620f6" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:53.418430  8652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:40444
I20260709 13:02:53.419175  8652 raft_consensus.cc:493] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:53.419683  8652 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.418442  8727 raft_consensus.cc:3060] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.426741  8651 leader_election.cc:290] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 2 election: Requested vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:53.427923  8801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3ba99ef29b47d2b480955461e620f6" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:53.428666  8801 raft_consensus.cc:3055] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:53.429014  8801 raft_consensus.cc:740] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:53.430013  8801 consensus_queue.cc:260] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.431255  8801 raft_consensus.cc:3060] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.431684  8727 raft_consensus.cc:2468] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 2.
I20260709 13:02:53.431465  8652 raft_consensus.cc:515] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.433187  8615 leader_election.cc:304] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:53.433946  8652 leader_election.cc:290] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 2 election: Requested vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:53.434406  8853 raft_consensus.cc:2804] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:53.434880  8853 raft_consensus.cc:697] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:53.435706  8853 consensus_queue.cc:237] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.438390  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:53.438848  8801 raft_consensus.cc:2468] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 2.
I20260709 13:02:53.439097  8726 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.439189  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:53.439994  8800 raft_consensus.cc:3055] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:53.440363  8800 raft_consensus.cc:740] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:53.441146  8800 consensus_queue.cc:260] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.443037  8876 consensus_queue.cc:1048] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.444553  8800 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.446524  8853 consensus_queue.cc:1048] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.448385  8547 catalog_manager.cc:5697] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 1 to 2, leader changed from 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67) to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:53.452916  8848 raft_consensus.cc:993] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3: : Instructing follower a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:53.453346  8848 raft_consensus.cc:1081] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Signalling peer a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:53.453955  8726 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 2.
I20260709 13:02:53.454900  8800 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 2.
I20260709 13:02:53.456033  8617 leader_election.cc:304] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:53.456830  8853 raft_consensus.cc:2804] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:53.457353  8853 raft_consensus.cc:697] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:53.458171  8853 consensus_queue.cc:237] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.458770  8904 consensus_queue.cc:1048] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.461061  8651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:40436
I20260709 13:02:53.461598  8651 raft_consensus.cc:493] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:53.461973  8651 raft_consensus.cc:3060] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.470703  8651 raft_consensus.cc:515] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.474453  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:53.475242  8726 raft_consensus.cc:3055] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:53.475865  8726 raft_consensus.cc:740] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:53.477048  8726 consensus_queue.cc:260] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.478610  8726 raft_consensus.cc:3060] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.483592  8922 consensus_queue.cc:1048] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:02:53.484036  8547 catalog_manager.cc:5697] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 1 to 2, leader changed from 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67) to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:53.486770  8651 leader_election.cc:290] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 2 election: Requested vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:53.488116  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:53.488692  8800 raft_consensus.cc:3060] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.496923  8726 raft_consensus.cc:2468] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 2.
I20260709 13:02:53.498104  8615 leader_election.cc:304] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:53.498551  8800 raft_consensus.cc:2468] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 2.
I20260709 13:02:53.499152  8922 raft_consensus.cc:2804] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:53.499676  8922 raft_consensus.cc:697] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:53.500672  8922 consensus_queue.cc:237] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.512895  8550 catalog_manager.cc:5697] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 1 to 2, leader changed from bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66) to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:53.578670  8863 consensus_queue.cc:1048] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.585819  8845 raft_consensus.cc:993] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517: : Instructing follower bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:02:53.587342  8906 raft_consensus.cc:1081] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Signalling peer bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:02:53.589224  8727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:02:53.590337  8727 raft_consensus.cc:493] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:53.590734  8727 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.592530  8906 consensus_queue.cc:1048] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:02:53.598201  8727 raft_consensus.cc:515] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.600540  8727 leader_election.cc:290] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 2 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:53.601400  8651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:53.602074  8651 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.602005  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:53.602820  8800 raft_consensus.cc:3055] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:53.603256  8800 raft_consensus.cc:740] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:53.604216  8800 consensus_queue.cc:260] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.605631  8800 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.609973  8651 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 2.
I20260709 13:02:53.610601  8904 raft_consensus.cc:993] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3: : Instructing follower 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:02:53.611153  8848 raft_consensus.cc:1081] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Signalling peer 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:02:53.612224  8800 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 2.
I20260709 13:02:53.612331  8691 leader_election.cc:304] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:53.613245  8876 raft_consensus.cc:2804] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:53.613185  8799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17b52764e3634a389db130e9f210681c"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:39600
I20260709 13:02:53.613915  8799 raft_consensus.cc:493] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:53.613847  8876 raft_consensus.cc:697] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Becoming Leader. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:53.614254  8799 raft_consensus.cc:3060] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.614758  8876 consensus_queue.cc:237] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.620533  8799 raft_consensus.cc:515] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.623044  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b52764e3634a389db130e9f210681c" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:53.622578  8549 catalog_manager.cc:5697] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 reported cstate change: term changed from 1 to 2, leader changed from 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67) to bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:53.623373  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b52764e3634a389db130e9f210681c" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:53.623662  8652 raft_consensus.cc:3060] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.623953  8727 raft_consensus.cc:3055] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:53.624258  8727 raft_consensus.cc:740] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:53.625360  8799 leader_election.cc:290] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 2 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:53.625073  8727 consensus_queue.cc:260] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.626170  8727 raft_consensus.cc:3060] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.630775  8652 raft_consensus.cc:2468] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 2.
I20260709 13:02:53.632004  8765 leader_election.cc:304] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:53.632843  8906 raft_consensus.cc:2804] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:53.633242  8906 raft_consensus.cc:697] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 2 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:53.633821  8906 consensus_queue.cc:237] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.634948  8727 raft_consensus.cc:2468] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 2.
I20260709 13:02:53.640812  8550 catalog_manager.cc:5697] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 1 to 2, leader changed from bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66) to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:53.668640  8863 raft_consensus.cc:993] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517: : Instructing follower bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:02:53.669092  8863 raft_consensus.cc:1081] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Signalling peer bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:02:53.670740  8727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09fe7a618b30456080341949aafde463"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:56216
I20260709 13:02:53.671252  8727 raft_consensus.cc:493] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:53.671562  8727 raft_consensus.cc:3060] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.677104  8727 raft_consensus.cc:515] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.678866  8727 leader_election.cc:290] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 2 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:53.679941  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:53.680217  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:53.680526  8652 raft_consensus.cc:3060] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.680883  8799 raft_consensus.cc:3055] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:53.681231  8799 raft_consensus.cc:740] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:53.681912  8799 consensus_queue.cc:260] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.682893  8799 raft_consensus.cc:3060] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:53.685846  8652 raft_consensus.cc:2468] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 2.
I20260709 13:02:53.687211  8691 leader_election.cc:304] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:53.688187  8876 raft_consensus.cc:2804] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:53.688956  8876 raft_consensus.cc:697] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Becoming Leader. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:53.688983  8799 raft_consensus.cc:2468] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 2.
I20260709 13:02:53.690049  8876 consensus_queue.cc:237] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:53.697757  8550 catalog_manager.cc:5697] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 reported cstate change: term changed from 1 to 2, leader changed from 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67) to bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:53.849351  8799 raft_consensus.cc:1275] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:53.850647  8876 consensus_queue.cc:1048] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:53.858554  8652 raft_consensus.cc:1275] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:53.860409  8876 consensus_queue.cc:1048] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:53.864312  8727 raft_consensus.cc:1275] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:53.867167  8922 consensus_queue.cc:1048] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:53.875619  8799 raft_consensus.cc:1275] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:53.877198  8922 consensus_queue.cc:1048] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:53.891639  8799 raft_consensus.cc:1275] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:53.892864  8922 consensus_queue.cc:1048] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:53.900832  8727 raft_consensus.cc:1275] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:53.902232  8927 consensus_queue.cc:1048] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:54.059381  8652 raft_consensus.cc:1275] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Refusing update from remote peer 7bafbb8448674ef6bad4f2343a88f517: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:54.060766  8863 consensus_queue.cc:1048] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:54.069116  8727 raft_consensus.cc:1275] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Refusing update from remote peer 7bafbb8448674ef6bad4f2343a88f517: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:54.070940  8862 consensus_queue.cc:1048] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:54.094051  8727 raft_consensus.cc:1275] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:54.095418  8857 consensus_queue.cc:1048] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:54.106496  8799 raft_consensus.cc:1275] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:54.107905  8857 consensus_queue.cc:1048] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:54.113407  8652 raft_consensus.cc:1275] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:54.114931  8904 consensus_queue.cc:1048] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:54.120491  8799 raft_consensus.cc:1275] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:54.122247  8848 consensus_queue.cc:1048] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:54.157217  8652 raft_consensus.cc:1275] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:54.158703  8876 consensus_queue.cc:1048] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:54.159584  8799 raft_consensus.cc:1275] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:54.160950  8876 consensus_queue.cc:1048] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:56.339214  8021 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:02:56.340058  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:56.357228  8652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
mode: GRACEFUL
new_leader_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:40452
I20260709 13:02:56.357800  8652 raft_consensus.cc:606] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 1 LEADER]: Received request to transfer leadership to TS 7bafbb8448674ef6bad4f2343a88f517
I20260709 13:02:56.360885  8652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
mode: GRACEFUL
new_leader_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:40452
I20260709 13:02:56.361583  8652 raft_consensus.cc:606] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Received request to transfer leadership to TS 7bafbb8448674ef6bad4f2343a88f517
I20260709 13:02:56.362787  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:02:56.363289  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:56.383889  8927 raft_consensus.cc:993] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169: : Instructing follower 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:02:56.384398  8929 raft_consensus.cc:1081] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Signalling peer 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:02:56.386014  8800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:39614
I20260709 13:02:56.386529  8800 raft_consensus.cc:493] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:56.386801  8800 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:56.391655  8800 raft_consensus.cc:515] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:56.393460  8800 leader_election.cc:290] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 3 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:56.394245  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:56.394481  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:56.394992  8652 raft_consensus.cc:3055] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:56.395141  8726 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:56.395334  8652 raft_consensus.cc:740] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:56.396334  8652 consensus_queue.cc:260] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:56.397606  8652 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:56.401204  8726 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 3.
I20260709 13:02:56.402349  8764 leader_election.cc:304] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:56.403129  8928 raft_consensus.cc:2804] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:56.403549  8652 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 3.
I20260709 13:02:56.404345  8928 raft_consensus.cc:697] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 3 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:56.405215  8928 consensus_queue.cc:237] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:56.411809  8547 catalog_manager.cc:5697] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 2 to 3, leader changed from a30440ab67344ff5bdb79368d2c87169 (127.7.213.65) to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 3 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:56.596335  8927 raft_consensus.cc:993] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169: : Instructing follower 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:02:56.596881  8929 raft_consensus.cc:1081] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 1 LEADER]: Signalling peer 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:02:56.598580  8800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:39614
I20260709 13:02:56.599185  8800 raft_consensus.cc:493] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:56.599581  8800 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:56.605674  8800 raft_consensus.cc:515] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:56.607547  8800 leader_election.cc:290] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 2 election: Requested vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827)
I20260709 13:02:56.608330  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:56.608700  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:02:56.608882  8726 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:56.609266  8652 raft_consensus.cc:3055] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:56.609556  8652 raft_consensus.cc:740] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:56.610611  8652 consensus_queue.cc:260] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:56.611820  8652 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:56.613674  8726 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 2.
I20260709 13:02:56.614812  8764 leader_election.cc:304] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:56.615602  8928 raft_consensus.cc:2804] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:56.616111  8928 raft_consensus.cc:697] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:56.616918  8928 consensus_queue.cc:237] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:56.618357  8652 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 2.
I20260709 13:02:56.624006  8547 catalog_manager.cc:5697] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 1 to 2, leader changed from a30440ab67344ff5bdb79368d2c87169 (127.7.213.65) to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:56.914179  8652 raft_consensus.cc:1275] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Refusing update from remote peer 7bafbb8448674ef6bad4f2343a88f517: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:56.915498  8928 consensus_queue.cc:1048] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:56.927345  8726 raft_consensus.cc:1275] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Refusing update from remote peer 7bafbb8448674ef6bad4f2343a88f517: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:56.932518  8933 consensus_queue.cc:1048] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:57.127789  8726 raft_consensus.cc:1275] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Refusing update from remote peer 7bafbb8448674ef6bad4f2343a88f517: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:57.129031  8942 consensus_queue.cc:1048] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:57.139889  8652 raft_consensus.cc:1275] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Refusing update from remote peer 7bafbb8448674ef6bad4f2343a88f517: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:57.141268  8933 consensus_queue.cc:1048] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:57.364104  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:57.374819  8800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
mode: GRACEFUL
new_leader_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:39630
I20260709 13:02:57.375422  8800 raft_consensus.cc:606] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 LEADER]: Received request to transfer leadership to TS a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:57.387401  8800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
mode: GRACEFUL
new_leader_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:39630
I20260709 13:02:57.388115  8800 raft_consensus.cc:606] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 3 LEADER]: Received request to transfer leadership to TS a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:57.396680  8727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
mode: GRACEFUL
new_leader_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:56246
I20260709 13:02:57.397372  8727 raft_consensus.cc:606] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Received request to transfer leadership to TS a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:57.402661  8727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09fe7a618b30456080341949aafde463"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
mode: GRACEFUL
new_leader_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:56246
I20260709 13:02:57.403291  8727 raft_consensus.cc:606] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Received request to transfer leadership to TS a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:57.409180  8727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
mode: GRACEFUL
new_leader_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:56246
I20260709 13:02:57.409703  8727 raft_consensus.cc:606] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Received request to transfer leadership to TS a30440ab67344ff5bdb79368d2c87169
I20260709 13:02:57.680737  8942 raft_consensus.cc:993] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517: : Instructing follower a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.681217  8933 raft_consensus.cc:1081] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 LEADER]: Signalling peer a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.682999  8652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:40444
I20260709 13:02:57.683571  8652 raft_consensus.cc:493] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:57.683944  8652 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.687917  8652 raft_consensus.cc:515] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:57.689227  8652 leader_election.cc:290] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 3 election: Requested vote from peers 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:02:57.690205  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:57.690510  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:57.690719  8800 raft_consensus.cc:3055] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:57.691046  8727 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.691056  8800 raft_consensus.cc:740] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:57.692035  8800 consensus_queue.cc:260] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:57.692950  8800 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.696147  8727 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 3.
I20260709 13:02:57.697438  8615 leader_election.cc:304] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:57.697727  8800 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 3.
I20260709 13:02:57.698318  8929 raft_consensus.cc:2804] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:57.699548  8929 raft_consensus.cc:697] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:57.700362  8929 consensus_queue.cc:237] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:02:57.707472  8949 raft_consensus.cc:993] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3: : Instructing follower a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.708125  8876 raft_consensus.cc:1081] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Signalling peer a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.708634  8547 catalog_manager.cc:5697] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 2 to 3, leader changed from 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67) to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:57.709905  8652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09fe7a618b30456080341949aafde463"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:40436
I20260709 13:02:57.710487  8652 raft_consensus.cc:493] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:57.710846  8652 raft_consensus.cc:3060] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.716835  8652 raft_consensus.cc:515] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.719971  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:57.719355  8652 leader_election.cc:290] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 3 election: Requested vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:57.720625  8800 raft_consensus.cc:3060] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.719878  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09fe7a618b30456080341949aafde463" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:57.722664  8727 raft_consensus.cc:3055] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:57.722983  8727 raft_consensus.cc:740] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:57.723590  8727 consensus_queue.cc:260] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.724700  8727 raft_consensus.cc:3060] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.726483  8800 raft_consensus.cc:2468] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 3.
I20260709 13:02:57.727584  8617 leader_election.cc:304] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:57.728677  8929 raft_consensus.cc:2804] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:57.729212  8929 raft_consensus.cc:697] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:57.730142  8929 consensus_queue.cc:237] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.731580  8727 raft_consensus.cc:2468] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 3.
I20260709 13:02:57.738416  8547 catalog_manager.cc:5697] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 2 to 3, leader changed from bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66) to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:57.835606  8876 raft_consensus.cc:993] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3: : Instructing follower a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.836231  8949 raft_consensus.cc:1081] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Signalling peer a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.837844  8652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:40436
I20260709 13:02:57.838378  8652 raft_consensus.cc:493] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:57.838681  8652 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.842725  8652 raft_consensus.cc:515] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.844278  8652 leader_election.cc:290] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 3 election: Requested vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:57.845263  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:57.845407  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:57.845808  8727 raft_consensus.cc:3055] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:02:57.845968  8799 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.846138  8727 raft_consensus.cc:740] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:57.846982  8727 consensus_queue.cc:260] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.847975  8727 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:02:57.852010  8799 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 3.
I20260709 13:02:57.853113  8617 leader_election.cc:304] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:57.853953  8929 raft_consensus.cc:2804] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:02:57.854009  8727 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 3.
I20260709 13:02:57.854465  8929 raft_consensus.cc:697] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:57.855268  8929 consensus_queue.cc:237] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.862437  8547 catalog_manager.cc:5697] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 2 to 3, leader changed from bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66) to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:57.946766  8876 raft_consensus.cc:993] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3: : Instructing follower a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.947255  8949 raft_consensus.cc:1081] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Signalling peer a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.948776  8652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:40436
I20260709 13:02:57.949445  8652 raft_consensus.cc:493] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:57.949776  8652 raft_consensus.cc:3060] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:57.954509  8652 raft_consensus.cc:515] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.956418  8652 leader_election.cc:290] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 2 election: Requested vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:57.957604  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:57.957531  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:57.958278  8799 raft_consensus.cc:3060] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:57.958242  8727 raft_consensus.cc:3055] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:02:57.958890  8727 raft_consensus.cc:740] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:02:57.959790  8727 consensus_queue.cc:260] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.961107  8727 raft_consensus.cc:3060] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:02:57.965754  8799 raft_consensus.cc:2468] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 2.
I20260709 13:02:57.967008  8617 leader_election.cc:304] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, a30440ab67344ff5bdb79368d2c87169; no voters: 
I20260709 13:02:57.967928  8929 raft_consensus.cc:2804] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:02:57.968273  8727 raft_consensus.cc:2468] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 2.
I20260709 13:02:57.968364  8929 raft_consensus.cc:697] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:57.969368  8929 consensus_queue.cc:237] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.975864  8942 raft_consensus.cc:993] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517: : Instructing follower a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.976285  8933 raft_consensus.cc:1081] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 3 LEADER]: Signalling peer a30440ab67344ff5bdb79368d2c87169 to start an election
I20260709 13:02:57.976662  8547 catalog_manager.cc:5697] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 1 to 2, leader changed from bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66) to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:57.977881  8652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
 from {username='slave'} at 127.0.0.1:40444
I20260709 13:02:57.978366  8652 raft_consensus.cc:493] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:57.978662  8652 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:57.984011  8652 raft_consensus.cc:515] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.985513  8652 leader_election.cc:290] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 4 election: Requested vote from peers bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:02:57.986393  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:02:57.986708  8799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a867b65c0da84df8b2f15ea4645de74b" candidate_uuid: "a30440ab67344ff5bdb79368d2c87169" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:02:57.986975  8727 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:57.987327  8799 raft_consensus.cc:3055] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:02:57.987586  8799 raft_consensus.cc:740] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:02:57.988314  8799 consensus_queue.cc:260] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:57.989179  8799 raft_consensus.cc:3060] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:02:57.992842  8727 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 4.
I20260709 13:02:57.993920  8615 leader_election.cc:304] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a30440ab67344ff5bdb79368d2c87169, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:02:57.994598  8929 raft_consensus.cc:2804] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:02:57.995029  8929 raft_consensus.cc:697] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 4 LEADER]: Becoming Leader. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:02:57.995373  8799 raft_consensus.cc:2468] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a30440ab67344ff5bdb79368d2c87169 in term 4.
I20260709 13:02:57.995821  8929 consensus_queue.cc:237] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:02:58.003378  8547 catalog_manager.cc:5697] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 reported cstate change: term changed from 3 to 4, leader changed from 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67) to a30440ab67344ff5bdb79368d2c87169 (127.7.213.65). New cstate: current_term: 4 leader_uuid: "a30440ab67344ff5bdb79368d2c87169" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:58.102772  8727 raft_consensus.cc:1275] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:58.104132  8929 consensus_queue.cc:1048] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:58.112921  8799 raft_consensus.cc:1275] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:58.114456  8929 consensus_queue.cc:1048] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:58.270512  8799 raft_consensus.cc:1275] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:58.272019  8929 consensus_queue.cc:1048] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:58.280948  8727 raft_consensus.cc:1275] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:58.282922  8948 consensus_queue.cc:1048] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:02:58.327059  8727 raft_consensus.cc:1275] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:58.328579  8948 consensus_queue.cc:1048] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:58.337296  8799 raft_consensus.cc:1275] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:02:58.338738  8962 consensus_queue.cc:1048] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:02:58.369732  8799 raft_consensus.cc:1275] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:58.371129  8962 consensus_queue.cc:1048] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:58.381095  8727 raft_consensus.cc:1275] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:02:58.382699  8948 consensus_queue.cc:1048] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:02:58.393114  8727 raft_consensus.cc:1275] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:58.394609  8948 consensus_queue.cc:1048] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:02:58.404078  8799 raft_consensus.cc:1275] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Refusing update from remote peer a30440ab67344ff5bdb79368d2c87169: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:02:58.405697  8948 consensus_queue.cc:1048] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:00.418304  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:00.428260  8652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
mode: GRACEFUL
new_leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:40452
I20260709 13:03:00.429282  8652 raft_consensus.cc:606] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Received request to transfer leadership to TS bb841c52fb5b41288d1dd2a9967e07b3
I20260709 13:03:00.432612  8652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
mode: GRACEFUL
new_leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:40452
I20260709 13:03:00.433185  8652 raft_consensus.cc:606] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Received request to transfer leadership to TS bb841c52fb5b41288d1dd2a9967e07b3
I20260709 13:03:00.435822  8652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
mode: GRACEFUL
new_leader_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:40452
I20260709 13:03:00.436385  8652 raft_consensus.cc:606] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Received request to transfer leadership to TS 7bafbb8448674ef6bad4f2343a88f517
I20260709 13:03:00.438807  8652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994"
dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
mode: GRACEFUL
new_leader_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:40452
I20260709 13:03:00.439374  8652 raft_consensus.cc:606] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Received request to transfer leadership to TS 7bafbb8448674ef6bad4f2343a88f517
I20260709 13:03:00.440831  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:03:00.441460  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:00.458673  8972 raft_consensus.cc:993] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169: : Instructing follower 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:03:00.459237  8962 raft_consensus.cc:1081] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Signalling peer 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:03:00.461126  8800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:39614
I20260709 13:03:00.461697  8800 raft_consensus.cc:493] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:00.462080  8800 raft_consensus.cc:3060] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.468267  8800 raft_consensus.cc:515] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:03:00.470211  8800 leader_election.cc:290] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 3 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:03:00.471148  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:03:00.471972  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db0a9299b66140fb91df4d8d4d329994" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:03:00.472131  8652 raft_consensus.cc:3055] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.472572  8727 raft_consensus.cc:3060] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.472529  8652 raft_consensus.cc:740] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:03:00.473578  8652 consensus_queue.cc:260] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:03:00.474982  8652 raft_consensus.cc:3060] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.480794  8727 raft_consensus.cc:2468] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 3.
I20260709 13:03:00.482460  8764 leader_election.cc:304] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:03:00.482615  8652 raft_consensus.cc:2468] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 3.
I20260709 13:03:00.483516  8933 raft_consensus.cc:2804] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:00.484807  8933 raft_consensus.cc:697] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:03:00.485819  8933 consensus_queue.cc:237] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:03:00.493733  8547 catalog_manager.cc:5697] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 2 to 3, leader changed from a30440ab67344ff5bdb79368d2c87169 (127.7.213.65) to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 3 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:00.832202  8962 raft_consensus.cc:993] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169: : Instructing follower bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:03:00.832690  8962 raft_consensus.cc:1081] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Signalling peer bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:03:00.834403  8726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:56232
I20260709 13:03:00.834899  8726 raft_consensus.cc:493] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:00.835196  8726 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:00.840245  8726 raft_consensus.cc:515] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:03:00.842024  8726 leader_election.cc:290] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 4 election: Requested vote from peers 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241), a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827)
I20260709 13:03:00.843056  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:03:00.843355  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b944a2a26ec4833bdd2a5ae0c33ad94" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:03:00.843554  8800 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:00.843931  8652 raft_consensus.cc:3055] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:00.844206  8652 raft_consensus.cc:740] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:03:00.844823  8652 consensus_queue.cc:260] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:03:00.845919  8652 raft_consensus.cc:3060] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:00.848058  8800 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 4.
I20260709 13:03:00.849372  8692 leader_election.cc:304] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:03:00.850256  8949 raft_consensus.cc:2804] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:00.850602  8652 raft_consensus.cc:2468] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 4.
I20260709 13:03:00.851507  8949 raft_consensus.cc:697] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 LEADER]: Becoming Leader. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:03:00.852463  8949 consensus_queue.cc:237] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } }
I20260709 13:03:00.861148  8547 catalog_manager.cc:5697] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 reported cstate change: term changed from 3 to 4, leader changed from a30440ab67344ff5bdb79368d2c87169 (127.7.213.65) to bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:00.878904  8652 raft_consensus.cc:1275] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Refusing update from remote peer 7bafbb8448674ef6bad4f2343a88f517: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:00.880334  8933 consensus_queue.cc:1048] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:00.892532  8726 raft_consensus.cc:1275] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Refusing update from remote peer 7bafbb8448674ef6bad4f2343a88f517: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:00.894380  8933 consensus_queue.cc:1048] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:00.929286  8972 raft_consensus.cc:993] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169: : Instructing follower bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:03:00.929836  8962 raft_consensus.cc:1081] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Signalling peer bb841c52fb5b41288d1dd2a9967e07b3 to start an election
I20260709 13:03:00.931540  8726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e"
dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
 from {username='slave'} at 127.0.0.1:56232
I20260709 13:03:00.932101  8726 raft_consensus.cc:493] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:00.932484  8726 raft_consensus.cc:3060] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.937284  8726 raft_consensus.cc:515] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:03:00.938861  8726 leader_election.cc:290] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 3 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67:42241)
I20260709 13:03:00.939636  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:03:00.940030  8800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a860cd902da4320bb1adc6e56613d0e" candidate_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
I20260709 13:03:00.940160  8652 raft_consensus.cc:3055] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:00.940552  8652 raft_consensus.cc:740] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:03:00.940546  8800 raft_consensus.cc:3060] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.941277  8652 consensus_queue.cc:260] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:03:00.942112  8652 raft_consensus.cc:3060] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:00.945271  8800 raft_consensus.cc:2468] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 3.
I20260709 13:03:00.946201  8652 raft_consensus.cc:2468] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb841c52fb5b41288d1dd2a9967e07b3 in term 3.
I20260709 13:03:00.946288  8692 leader_election.cc:304] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:03:00.947162  8949 raft_consensus.cc:2804] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:00.947610  8949 raft_consensus.cc:697] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 LEADER]: Becoming Leader. State: Replica: bb841c52fb5b41288d1dd2a9967e07b3, State: Running, Role: LEADER
I20260709 13:03:00.948527  8949 consensus_queue.cc:237] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:03:00.955247  8549 catalog_manager.cc:5697] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 reported cstate change: term changed from 2 to 3, leader changed from a30440ab67344ff5bdb79368d2c87169 (127.7.213.65) to bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:01.030289  8962 raft_consensus.cc:993] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169: : Instructing follower 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:03:01.030769  8972 raft_consensus.cc:1081] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Signalling peer 7bafbb8448674ef6bad4f2343a88f517 to start an election
I20260709 13:03:01.032351  8800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc"
dest_uuid: "7bafbb8448674ef6bad4f2343a88f517"
 from {username='slave'} at 127.0.0.1:39614
I20260709 13:03:01.032837  8800 raft_consensus.cc:493] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:01.033149  8800 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:01.037257  8800 raft_consensus.cc:515] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:03:01.038729  8800 leader_election.cc:290] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 4 election: Requested vote from peers a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827), bb841c52fb5b41288d1dd2a9967e07b3 (127.7.213.66:39009)
I20260709 13:03:01.039680  8652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a30440ab67344ff5bdb79368d2c87169"
I20260709 13:03:01.039839  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c20d8be02da4e3387df40c8466193dc" candidate_uuid: "7bafbb8448674ef6bad4f2343a88f517" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bb841c52fb5b41288d1dd2a9967e07b3"
I20260709 13:03:01.040300  8652 raft_consensus.cc:3055] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:01.040378  8726 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:01.040629  8652 raft_consensus.cc:740] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a30440ab67344ff5bdb79368d2c87169, State: Running, Role: LEADER
I20260709 13:03:01.041406  8652 consensus_queue.cc:260] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:03:01.042277  8652 raft_consensus.cc:3060] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:01.044771  8726 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 4.
I20260709 13:03:01.046028  8764 leader_election.cc:304] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bafbb8448674ef6bad4f2343a88f517, bb841c52fb5b41288d1dd2a9967e07b3; no voters: 
I20260709 13:03:01.047012  8933 raft_consensus.cc:2804] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:01.047263  8652 raft_consensus.cc:2468] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bafbb8448674ef6bad4f2343a88f517 in term 4.
I20260709 13:03:01.047557  8933 raft_consensus.cc:697] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 4 LEADER]: Becoming Leader. State: Replica: 7bafbb8448674ef6bad4f2343a88f517, State: Running, Role: LEADER
I20260709 13:03:01.048522  8933 consensus_queue.cc:237] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } }
I20260709 13:03:01.056591  8549 catalog_manager.cc:5697] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 reported cstate change: term changed from 3 to 4, leader changed from a30440ab67344ff5bdb79368d2c87169 (127.7.213.65) to 7bafbb8448674ef6bad4f2343a88f517 (127.7.213.67). New cstate: current_term: 4 leader_uuid: "7bafbb8448674ef6bad4f2343a88f517" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.233465  8800 raft_consensus.cc:1275] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:01.234776  8949 consensus_queue.cc:1048] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:01.244817  8652 raft_consensus.cc:1275] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:01.246872  8990 consensus_queue.cc:1048] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:01.343276  8800 raft_consensus.cc:1275] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:01.344691  8990 consensus_queue.cc:1048] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bafbb8448674ef6bad4f2343a88f517" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:01.354547  8652 raft_consensus.cc:1275] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Refusing update from remote peer bb841c52fb5b41288d1dd2a9967e07b3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:01.355997  8979 consensus_queue.cc:1048] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a30440ab67344ff5bdb79368d2c87169" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:01.442341  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:01.448321  8021 tablet_server.cc:179] TabletServer@127.7.213.65:0 shutting down...
W20260709 13:03:01.453485  8765 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 13:03:01.457950  8765 consensus_peers.cc:597] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 -> Peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Couldn't send request to peer a30440ab67344ff5bdb79368d2c87169. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:01.476543  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:01.477474  8021 tablet_replica.cc:333] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.478348  8021 raft_consensus.cc:2243] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.478785  8021 raft_consensus.cc:2272] T 7c20d8be02da4e3387df40c8466193dc P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.481241  8021 tablet_replica.cc:333] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.481825  8021 raft_consensus.cc:2243] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.482322  8021 raft_consensus.cc:2272] T a67e4eb9c5634896a1840bb1c1fb134d P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.484315  8021 tablet_replica.cc:333] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.484978  8021 raft_consensus.cc:2243] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.485761  8021 raft_consensus.cc:2272] T a867b65c0da84df8b2f15ea4645de74b P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.487658  8021 tablet_replica.cc:333] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.488334  8021 raft_consensus.cc:2243] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.488844  8021 raft_consensus.cc:2272] T 17b52764e3634a389db130e9f210681c P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.490854  8021 tablet_replica.cc:333] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.491402  8021 raft_consensus.cc:2243] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.491921  8021 raft_consensus.cc:2272] T 4a860cd902da4320bb1adc6e56613d0e P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.494138  8021 tablet_replica.cc:333] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.494680  8021 raft_consensus.cc:2243] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.495481  8021 raft_consensus.cc:2272] T 62ada425b0dc451293343153da445ec1 P a30440ab67344ff5bdb79368d2c87169 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.497387  8021 tablet_replica.cc:333] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.498000  8021 raft_consensus.cc:2243] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.498811  8021 raft_consensus.cc:2272] T 09fe7a618b30456080341949aafde463 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.500972  8021 tablet_replica.cc:333] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.501518  8021 raft_consensus.cc:2243] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.502343  8021 raft_consensus.cc:2272] T 3f3ba99ef29b47d2b480955461e620f6 P a30440ab67344ff5bdb79368d2c87169 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.504235  8021 tablet_replica.cc:333] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.504758  8021 raft_consensus.cc:2243] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.505290  8021 raft_consensus.cc:2272] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P a30440ab67344ff5bdb79368d2c87169 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.508360  8021 tablet_replica.cc:333] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169: stopping tablet replica
I20260709 13:03:01.508579  8726 raft_consensus.cc:1275] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Refusing update from remote peer 7bafbb8448674ef6bad4f2343a88f517: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:01.508924  8021 raft_consensus.cc:2243] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.509393  8021 raft_consensus.cc:2272] T db0a9299b66140fb91df4d8d4d329994 P a30440ab67344ff5bdb79368d2c87169 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.509917  8933 consensus_queue.cc:1048] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb841c52fb5b41288d1dd2a9967e07b3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 39009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:03:01.521093  8765 consensus_peers.cc:597] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 -> Peer a30440ab67344ff5bdb79368d2c87169 (127.7.213.65:41827): Couldn't send request to peer a30440ab67344ff5bdb79368d2c87169. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:01.538475  8021 tablet_server.cc:196] TabletServer@127.7.213.65:0 shutdown complete.
I20260709 13:03:01.562564  8021 tablet_server.cc:179] TabletServer@127.7.213.66:0 shutting down...
I20260709 13:03:01.589121  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:01.589974  8021 tablet_replica.cc:333] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.590629  8021 raft_consensus.cc:2243] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.591408  8021 raft_consensus.cc:2272] T a67e4eb9c5634896a1840bb1c1fb134d P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.593389  8021 tablet_replica.cc:333] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.593943  8021 raft_consensus.cc:2243] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.594331  8021 raft_consensus.cc:2272] T 09fe7a618b30456080341949aafde463 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.596138  8021 tablet_replica.cc:333] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.596648  8021 raft_consensus.cc:2243] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.597095  8021 raft_consensus.cc:2272] T db0a9299b66140fb91df4d8d4d329994 P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.600208  8021 tablet_replica.cc:333] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.600772  8021 raft_consensus.cc:2243] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.601193  8021 raft_consensus.cc:2272] T 62ada425b0dc451293343153da445ec1 P bb841c52fb5b41288d1dd2a9967e07b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.602957  8021 tablet_replica.cc:333] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.603511  8021 raft_consensus.cc:2243] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.604377  8021 raft_consensus.cc:2272] T 4a860cd902da4320bb1adc6e56613d0e P bb841c52fb5b41288d1dd2a9967e07b3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.607395  8021 tablet_replica.cc:333] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.608093  8021 raft_consensus.cc:2243] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.608590  8021 raft_consensus.cc:2272] T 17b52764e3634a389db130e9f210681c P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.610551  8021 tablet_replica.cc:333] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.611090  8021 raft_consensus.cc:2243] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.611945  8021 raft_consensus.cc:2272] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.614333  8021 tablet_replica.cc:333] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.614889  8021 raft_consensus.cc:2243] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.615412  8021 raft_consensus.cc:2272] T 3f3ba99ef29b47d2b480955461e620f6 P bb841c52fb5b41288d1dd2a9967e07b3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.617638  8021 tablet_replica.cc:333] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.618151  8021 raft_consensus.cc:2243] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.618607  8021 raft_consensus.cc:2272] T 7c20d8be02da4e3387df40c8466193dc P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.620899  8021 tablet_replica.cc:333] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3: stopping tablet replica
I20260709 13:03:01.621417  8021 raft_consensus.cc:2243] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.621867  8021 raft_consensus.cc:2272] T a867b65c0da84df8b2f15ea4645de74b P bb841c52fb5b41288d1dd2a9967e07b3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.651484  8021 tablet_server.cc:196] TabletServer@127.7.213.66:0 shutdown complete.
I20260709 13:03:01.669660  8021 tablet_server.cc:179] TabletServer@127.7.213.67:0 shutting down...
I20260709 13:03:01.694624  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:01.695364  8021 tablet_replica.cc:333] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.696350  8021 raft_consensus.cc:2243] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.697263  8021 raft_consensus.cc:2272] T db0a9299b66140fb91df4d8d4d329994 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.699520  8021 tablet_replica.cc:333] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.700235  8021 raft_consensus.cc:2243] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.700680  8021 raft_consensus.cc:2272] T a67e4eb9c5634896a1840bb1c1fb134d P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.702358  8021 tablet_replica.cc:333] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.702798  8021 raft_consensus.cc:2243] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.703151  8021 raft_consensus.cc:2272] T a867b65c0da84df8b2f15ea4645de74b P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.704730  8021 tablet_replica.cc:333] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.705159  8021 raft_consensus.cc:2243] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.705874  8021 raft_consensus.cc:2272] T 7c20d8be02da4e3387df40c8466193dc P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.707727  8021 tablet_replica.cc:333] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.708237  8021 raft_consensus.cc:2243] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.708920  8021 raft_consensus.cc:2272] T 17b52764e3634a389db130e9f210681c P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.710454  8021 tablet_replica.cc:333] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.710872  8021 raft_consensus.cc:2243] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.711200  8021 raft_consensus.cc:2272] T 62ada425b0dc451293343153da445ec1 P 7bafbb8448674ef6bad4f2343a88f517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.712785  8021 tablet_replica.cc:333] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.713202  8021 raft_consensus.cc:2243] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.713532  8021 raft_consensus.cc:2272] T 4a860cd902da4320bb1adc6e56613d0e P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.715425  8021 tablet_replica.cc:333] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.715971  8021 raft_consensus.cc:2243] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.716368  8021 raft_consensus.cc:2272] T 3f3ba99ef29b47d2b480955461e620f6 P 7bafbb8448674ef6bad4f2343a88f517 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.717932  8021 tablet_replica.cc:333] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.718336  8021 raft_consensus.cc:2243] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.718688  8021 raft_consensus.cc:2272] T 3b944a2a26ec4833bdd2a5ae0c33ad94 P 7bafbb8448674ef6bad4f2343a88f517 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.721235  8021 tablet_replica.cc:333] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517: stopping tablet replica
I20260709 13:03:01.721777  8021 raft_consensus.cc:2243] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:01.722312  8021 raft_consensus.cc:2272] T 09fe7a618b30456080341949aafde463 P 7bafbb8448674ef6bad4f2343a88f517 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.751837  8021 tablet_server.cc:196] TabletServer@127.7.213.67:0 shutdown complete.
I20260709 13:03:01.771641  8021 master.cc:562] Master@127.7.213.126:40949 shutting down...
I20260709 13:03:01.794479  8021 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:01.794982  8021 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:01.795260  8021 tablet_replica.cc:333] T 00000000000000000000000000000000 P 76d72720e986447b8a0e2fb653dda6e9: stopping tablet replica
I20260709 13:03:01.815654  8021 master.cc:584] Master@127.7.213.126:40949 shutdown complete.
7bafbb8448674ef6bad4f2343a88f517  1
a30440ab67344ff5bdb79368d2c87169  5
bb841c52fb5b41288d1dd2a9967e07b3  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10485 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:03:01.856302  8021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.213.126:45141
I20260709 13:03:01.857470  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:01.863638  8996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:01.864009  8997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:01.864953  8999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:01.866182  8021 server_base.cc:1061] running on GCE node
I20260709 13:03:01.867061  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:01.867239  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:01.867421  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602181867407 us; error 0 us; skew 500 ppm
I20260709 13:03:01.868062  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:01.870537  8021 webserver.cc:533] Webserver started at http://127.7.213.126:45545/ using document root <none> and password file <none>
I20260709 13:03:01.871040  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:01.871227  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:01.871526  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:01.872709  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/master-0-root/instance:
uuid: "cdc5c504a10a4da1812e571d14ae3abd"
format_stamp: "Formatted at 2026-07-09 13:03:01 on dist-test-slave-9g29"
I20260709 13:03:01.877637  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:03:01.881259  9004 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:01.882225  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:03:01.882611  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/master-0-root
uuid: "cdc5c504a10a4da1812e571d14ae3abd"
format_stamp: "Formatted at 2026-07-09 13:03:01 on dist-test-slave-9g29"
I20260709 13:03:01.882949  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:01.903703  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:01.905179  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:01.945194  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.126:45141
I20260709 13:03:01.945315  9055 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.126:45141 every 8 connection(s)
I20260709 13:03:01.949165  9056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:01.959234  9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Bootstrap starting.
I20260709 13:03:01.964344  9056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:01.969074  9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: No bootstrap required, opened a new log
I20260709 13:03:01.971484  9056 raft_consensus.cc:359] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5c504a10a4da1812e571d14ae3abd" member_type: VOTER }
I20260709 13:03:01.971987  9056 raft_consensus.cc:385] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:01.972281  9056 raft_consensus.cc:740] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdc5c504a10a4da1812e571d14ae3abd, State: Initialized, Role: FOLLOWER
I20260709 13:03:01.973004  9056 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5c504a10a4da1812e571d14ae3abd" member_type: VOTER }
I20260709 13:03:01.973542  9056 raft_consensus.cc:399] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:03:01.973801  9056 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:03:01.974068  9056 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:01.978888  9056 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5c504a10a4da1812e571d14ae3abd" member_type: VOTER }
I20260709 13:03:01.979504  9056 leader_election.cc:304] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdc5c504a10a4da1812e571d14ae3abd; no voters: 
I20260709 13:03:01.980837  9056 leader_election.cc:290] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:03:01.981251  9059 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:01.982770  9059 raft_consensus.cc:697] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 1 LEADER]: Becoming Leader. State: Replica: cdc5c504a10a4da1812e571d14ae3abd, State: Running, Role: LEADER
I20260709 13:03:01.983512  9059 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5c504a10a4da1812e571d14ae3abd" member_type: VOTER }
I20260709 13:03:01.984385  9056 sys_catalog.cc:565] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:03:01.989068  9061 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdc5c504a10a4da1812e571d14ae3abd. Latest consensus state: current_term: 1 leader_uuid: "cdc5c504a10a4da1812e571d14ae3abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5c504a10a4da1812e571d14ae3abd" member_type: VOTER } }
I20260709 13:03:01.989616  9060 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdc5c504a10a4da1812e571d14ae3abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc5c504a10a4da1812e571d14ae3abd" member_type: VOTER } }
I20260709 13:03:01.989996  9060 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:01.989916  9061 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:01.991998  9065 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:03:01.997030  9065 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:03:02.003942  8021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:03:02.005774  9065 catalog_manager.cc:1383] Generated new cluster ID: 1d2bdd7d7e3844b08ee4609d11c49a59
I20260709 13:03:02.006090  9065 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:03:02.023541  9065 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:03:02.024881  9065 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:03:02.036806  9065 catalog_manager.cc:6070] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Generated new TSK 0
I20260709 13:03:02.037456  9065 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:03:02.071337  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:02.077778  9077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:02.079710  9078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:02.080857  9080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:02.081465  8021 server_base.cc:1061] running on GCE node
I20260709 13:03:02.082319  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:02.082494  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:02.082702  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602182082668 us; error 0 us; skew 500 ppm
I20260709 13:03:02.083230  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:02.086028  8021 webserver.cc:533] Webserver started at http://127.7.213.65:40615/ using document root <none> and password file <none>
I20260709 13:03:02.086548  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:02.086732  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:02.087010  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:02.088119  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/instance:
uuid: "ce51b61da22f40d4aa0204a6b1028f28"
format_stamp: "Formatted at 2026-07-09 13:03:02 on dist-test-slave-9g29"
I20260709 13:03:02.092832  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:03:02.096292  9085 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:02.097275  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:03:02.097591  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "ce51b61da22f40d4aa0204a6b1028f28"
format_stamp: "Formatted at 2026-07-09 13:03:02 on dist-test-slave-9g29"
I20260709 13:03:02.097849  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:02.110879  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:02.112200  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:02.113663  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:02.115938  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:02.116186  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:02.116431  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:02.116634  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:02.159142  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:39939
I20260709 13:03:02.159225  9147 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:39939 every 8 connection(s)
I20260709 13:03:02.164184  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:02.171154  9152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:02.172235  9153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:02.175571  8021 server_base.cc:1061] running on GCE node
W20260709 13:03:02.175529  9155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:02.176774  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:02.177004  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:02.177188  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602182177172 us; error 0 us; skew 500 ppm
I20260709 13:03:02.177701  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:02.180335  8021 webserver.cc:533] Webserver started at http://127.7.213.66:37197/ using document root <none> and password file <none>
I20260709 13:03:02.180938  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:02.181200  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:02.181537  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:02.182979  9148 heartbeater.cc:344] Connected to a master server at 127.7.213.126:45141
I20260709 13:03:02.182916  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/instance:
uuid: "94504d29c6744f38bb06943479df9466"
format_stamp: "Formatted at 2026-07-09 13:03:02 on dist-test-slave-9g29"
I20260709 13:03:02.183393  9148 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:02.184327  9148 heartbeater.cc:507] Master 127.7.213.126:45141 requested a full tablet report, sending...
I20260709 13:03:02.186906  9021 ts_manager.cc:194] Registered new tserver with Master: ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:02.189069  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:03:02.189241  9021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36610
I20260709 13:03:02.193432  9160 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:02.194234  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:03:02.194504  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root
uuid: "94504d29c6744f38bb06943479df9466"
format_stamp: "Formatted at 2026-07-09 13:03:02 on dist-test-slave-9g29"
I20260709 13:03:02.194759  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:02.209733  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:02.210886  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:02.212335  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:02.214825  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:02.215005  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:02.215440  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:02.215592  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:02.259156  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.66:34417
I20260709 13:03:02.259256  9222 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.66:34417 every 8 connection(s)
I20260709 13:03:02.264125  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:02.271675  9226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:02.273512  9227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:02.275120  9229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:02.275971  8021 server_base.cc:1061] running on GCE node
I20260709 13:03:02.276780  9223 heartbeater.cc:344] Connected to a master server at 127.7.213.126:45141
I20260709 13:03:02.276932  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:03:02.277141  9223 heartbeater.cc:461] Registering TS with master...
W20260709 13:03:02.277256  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:02.277571  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602182277552 us; error 0 us; skew 500 ppm
I20260709 13:03:02.278126  9223 heartbeater.cc:507] Master 127.7.213.126:45141 requested a full tablet report, sending...
I20260709 13:03:02.278218  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:02.280339  9021 ts_manager.cc:194] Registered new tserver with Master: 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:02.281049  8021 webserver.cc:533] Webserver started at http://127.7.213.67:39515/ using document root <none> and password file <none>
I20260709 13:03:02.281584  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:02.281791  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:02.281985  9021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36626
I20260709 13:03:02.282084  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:02.283664  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/instance:
uuid: "1efabcb1befc400aadf101e8276fc1aa"
format_stamp: "Formatted at 2026-07-09 13:03:02 on dist-test-slave-9g29"
I20260709 13:03:02.288909  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:03:02.292821  9234 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:02.293706  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:03:02.294031  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root
uuid: "1efabcb1befc400aadf101e8276fc1aa"
format_stamp: "Formatted at 2026-07-09 13:03:02 on dist-test-slave-9g29"
I20260709 13:03:02.294286  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:02.319463  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:02.320744  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:02.322356  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:02.324787  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:02.325035  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:02.325279  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:02.325457  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:02.366699  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.67:38807
I20260709 13:03:02.366809  9296 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.67:38807 every 8 connection(s)
I20260709 13:03:02.381403  9297 heartbeater.cc:344] Connected to a master server at 127.7.213.126:45141
I20260709 13:03:02.381788  9297 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:02.382467  9297 heartbeater.cc:507] Master 127.7.213.126:45141 requested a full tablet report, sending...
I20260709 13:03:02.384635  9021 ts_manager.cc:194] Registered new tserver with Master: 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:02.384764  8021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014917101s
I20260709 13:03:02.386304  9021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36634
I20260709 13:03:02.412557  9021 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36636:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:03:02.418738  9021 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:03:02.641427  9188 tablet_service.cc:1511] Processing CreateTablet for tablet e58cbd03222246299b7e4363d8fcffa6 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:02.643059  9188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58cbd03222246299b7e4363d8fcffa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.642730  9185 tablet_service.cc:1511] Processing CreateTablet for tablet 98d913b72e5b43e8b8f20fb99d0838c4 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:02.644935  9186 tablet_service.cc:1511] Processing CreateTablet for tablet ca0b93b18a7f4e6398018531838527d9 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:02.645494  9109 tablet_service.cc:1511] Processing CreateTablet for tablet bd9140502bd3469eb208753dea0f3159 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:02.645615  9113 tablet_service.cc:1511] Processing CreateTablet for tablet e58cbd03222246299b7e4363d8fcffa6 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:02.646958  9109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd9140502bd3469eb208753dea0f3159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.647663  9185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d913b72e5b43e8b8f20fb99d0838c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.647548  9110 tablet_service.cc:1511] Processing CreateTablet for tablet 98d913b72e5b43e8b8f20fb99d0838c4 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:02.649849  9184 tablet_service.cc:1511] Processing CreateTablet for tablet bd9140502bd3469eb208753dea0f3159 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:02.646958  9187 tablet_service.cc:1511] Processing CreateTablet for tablet 43e874ff0724409b88c5ec377b73a06a (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:02.650799  9111 tablet_service.cc:1511] Processing CreateTablet for tablet ca0b93b18a7f4e6398018531838527d9 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:02.652079  9112 tablet_service.cc:1511] Processing CreateTablet for tablet 43e874ff0724409b88c5ec377b73a06a (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:02.652673  9184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd9140502bd3469eb208753dea0f3159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.653563  9110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d913b72e5b43e8b8f20fb99d0838c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.656378  9111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca0b93b18a7f4e6398018531838527d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.660375  9187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e874ff0724409b88c5ec377b73a06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.664466  9186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca0b93b18a7f4e6398018531838527d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.668434  9112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e874ff0724409b88c5ec377b73a06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.674677  9113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58cbd03222246299b7e4363d8fcffa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.670930  9108 tablet_service.cc:1511] Processing CreateTablet for tablet d8f97d26e68b4fbead130c830dbf4b51 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:02.675477  9312 tablet_bootstrap.cc:492] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:02.677995  9183 tablet_service.cc:1511] Processing CreateTablet for tablet d8f97d26e68b4fbead130c830dbf4b51 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:02.679478  9183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f97d26e68b4fbead130c830dbf4b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.681075  9262 tablet_service.cc:1511] Processing CreateTablet for tablet e58cbd03222246299b7e4363d8fcffa6 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:02.687827  9262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58cbd03222246299b7e4363d8fcffa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.687951  9108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f97d26e68b4fbead130c830dbf4b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.689630  9258 tablet_service.cc:1511] Processing CreateTablet for tablet bd9140502bd3469eb208753dea0f3159 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:02.691176  9258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd9140502bd3469eb208753dea0f3159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.695043  9261 tablet_service.cc:1511] Processing CreateTablet for tablet 43e874ff0724409b88c5ec377b73a06a (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:02.696712  9261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e874ff0724409b88c5ec377b73a06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.697582  9260 tablet_service.cc:1511] Processing CreateTablet for tablet ca0b93b18a7f4e6398018531838527d9 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:02.697824  9109 tablet_service.cc:1511] Processing CreateTablet for tablet f84c9fc901364e60b2c414ee3b7a7e65 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:02.699059  9260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca0b93b18a7f4e6398018531838527d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.696707  9259 tablet_service.cc:1511] Processing CreateTablet for tablet 98d913b72e5b43e8b8f20fb99d0838c4 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:02.700203  9182 tablet_service.cc:1511] Processing CreateTablet for tablet f84c9fc901364e60b2c414ee3b7a7e65 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:02.700865  9259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d913b72e5b43e8b8f20fb99d0838c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.701608  9182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f84c9fc901364e60b2c414ee3b7a7e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.704906  9257 tablet_service.cc:1511] Processing CreateTablet for tablet d8f97d26e68b4fbead130c830dbf4b51 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:02.705169  9107 tablet_service.cc:1511] Processing CreateTablet for tablet a51a0795b52e4843959cd3be3d077713 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:02.706573  9257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f97d26e68b4fbead130c830dbf4b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.707818  9109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f84c9fc901364e60b2c414ee3b7a7e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.708508  9107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51a0795b52e4843959cd3be3d077713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.709187  9256 tablet_service.cc:1511] Processing CreateTablet for tablet f84c9fc901364e60b2c414ee3b7a7e65 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:02.710810  9256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f84c9fc901364e60b2c414ee3b7a7e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.726500  9181 tablet_service.cc:1511] Processing CreateTablet for tablet a51a0795b52e4843959cd3be3d077713 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:02.728583  9181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51a0795b52e4843959cd3be3d077713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.729768  9255 tablet_service.cc:1511] Processing CreateTablet for tablet a51a0795b52e4843959cd3be3d077713 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:02.730273  9110 tablet_service.cc:1511] Processing CreateTablet for tablet 1c199cae5ff14a478726c8a8b491f8f7 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:02.731194  9255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51a0795b52e4843959cd3be3d077713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.731639  9110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c199cae5ff14a478726c8a8b491f8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.735728  9313 tablet_bootstrap.cc:492] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:02.739063  9188 tablet_service.cc:1511] Processing CreateTablet for tablet 1c199cae5ff14a478726c8a8b491f8f7 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:02.741583  9254 tablet_service.cc:1511] Processing CreateTablet for tablet 1c199cae5ff14a478726c8a8b491f8f7 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:02.743669  9106 tablet_service.cc:1511] Processing CreateTablet for tablet ff8d71c72ec64addaf0593eca7fd672d (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:02.745080  9106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8d71c72ec64addaf0593eca7fd672d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.759004  9185 tablet_service.cc:1511] Processing CreateTablet for tablet ff8d71c72ec64addaf0593eca7fd672d (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:02.743593  9254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c199cae5ff14a478726c8a8b491f8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.763717  9253 tablet_service.cc:1511] Processing CreateTablet for tablet ff8d71c72ec64addaf0593eca7fd672d (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:02.765224  9253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8d71c72ec64addaf0593eca7fd672d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.784224  9188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c199cae5ff14a478726c8a8b491f8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.785053  9185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8d71c72ec64addaf0593eca7fd672d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.786180  9312 tablet_bootstrap.cc:654] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.786545  9105 tablet_service.cc:1511] Processing CreateTablet for tablet fa5405d9d7b34438b7f5732807fac54a (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:02.788141  9105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5405d9d7b34438b7f5732807fac54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.797668  9182 tablet_service.cc:1511] Processing CreateTablet for tablet fa5405d9d7b34438b7f5732807fac54a (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:02.816912  9182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5405d9d7b34438b7f5732807fac54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.822703  9256 tablet_service.cc:1511] Processing CreateTablet for tablet fa5405d9d7b34438b7f5732807fac54a (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:02.824265  9256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5405d9d7b34438b7f5732807fac54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.825260  9313 tablet_bootstrap.cc:654] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.851429  9257 tablet_service.cc:1511] Processing CreateTablet for tablet 776f24c8f05046a0a730da5e6e801cb7 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:02.852535  9314 tablet_bootstrap.cc:492] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:02.853005  9257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 776f24c8f05046a0a730da5e6e801cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.860013  9312 tablet_bootstrap.cc:492] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:02.860517  9312 ts_tablet_manager.cc:1403] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.185s	user 0.016s	sys 0.008s
I20260709 13:03:02.862361  9313 tablet_bootstrap.cc:492] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:02.862821  9313 ts_tablet_manager.cc:1403] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.127s	user 0.007s	sys 0.008s
I20260709 13:03:02.863488  9312 raft_consensus.cc:359] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:02.864251  9312 raft_consensus.cc:385] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.863966  9109 tablet_service.cc:1511] Processing CreateTablet for tablet 8d9ec1fba08a49aeb3e1a8a52e798291 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:02.864566  9312 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.865864  9109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9ec1fba08a49aeb3e1a8a52e798291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.866789  9187 tablet_service.cc:1511] Processing CreateTablet for tablet 776f24c8f05046a0a730da5e6e801cb7 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:02.868108  9187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 776f24c8f05046a0a730da5e6e801cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.869454  9183 tablet_service.cc:1511] Processing CreateTablet for tablet 8d9ec1fba08a49aeb3e1a8a52e798291 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:02.870903  9183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9ec1fba08a49aeb3e1a8a52e798291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.875537  9314 tablet_bootstrap.cc:654] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:02.883201  9312 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:02.886456  9110 tablet_service.cc:1511] Processing CreateTablet for tablet 991b28944986479a8ad096ca92f1a435 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:02.898037  9110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991b28944986479a8ad096ca92f1a435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.900096  9259 tablet_service.cc:1511] Processing CreateTablet for tablet 8d9ec1fba08a49aeb3e1a8a52e798291 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:02.901415  9259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9ec1fba08a49aeb3e1a8a52e798291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.901160  9258 tablet_service.cc:1511] Processing CreateTablet for tablet 991b28944986479a8ad096ca92f1a435 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:02.890149  9108 tablet_service.cc:1511] Processing CreateTablet for tablet 776f24c8f05046a0a730da5e6e801cb7 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:02.907648  9258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991b28944986479a8ad096ca92f1a435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.915372  9181 tablet_service.cc:1511] Processing CreateTablet for tablet 55cb8917093e4273bb3af2d4d36815cf (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:02.916956  9181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55cb8917093e4273bb3af2d4d36815cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.916862  9254 tablet_service.cc:1511] Processing CreateTablet for tablet 55cb8917093e4273bb3af2d4d36815cf (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:02.918946  9254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55cb8917093e4273bb3af2d4d36815cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.918720  9106 tablet_service.cc:1511] Processing CreateTablet for tablet 7597d602dc674623ae1d5de5fdb5d2ee (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:02.920300  9106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7597d602dc674623ae1d5de5fdb5d2ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.921797  9253 tablet_service.cc:1511] Processing CreateTablet for tablet 7597d602dc674623ae1d5de5fdb5d2ee (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:02.923619  9253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7597d602dc674623ae1d5de5fdb5d2ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.925405  9186 tablet_service.cc:1511] Processing CreateTablet for tablet 7597d602dc674623ae1d5de5fdb5d2ee (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:02.926775  9186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7597d602dc674623ae1d5de5fdb5d2ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.928071  9108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 776f24c8f05046a0a730da5e6e801cb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.937239  9105 tablet_service.cc:1511] Processing CreateTablet for tablet 55cb8917093e4273bb3af2d4d36815cf (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:02.901096  9185 tablet_service.cc:1511] Processing CreateTablet for tablet 991b28944986479a8ad096ca92f1a435 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:02.952142  9313 raft_consensus.cc:359] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:02.952651  9185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991b28944986479a8ad096ca92f1a435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.952955  9313 raft_consensus.cc:385] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.953269  9313 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:02.953497  9112 tablet_service.cc:1511] Processing CreateTablet for tablet 9daa912f9e08438caf063c0c83e09fe5 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:02.954047  9313 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:02.954888  9112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9daa912f9e08438caf063c0c83e09fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.960518  9105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55cb8917093e4273bb3af2d4d36815cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.972363  9312 ts_tablet_manager.cc:1434] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.112s	user 0.007s	sys 0.007s
I20260709 13:03:02.973430  9312 tablet_bootstrap.cc:492] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:02.973795  9223 heartbeater.cc:499] Master 127.7.213.126:45141 was elected leader, sending a full tablet report...
I20260709 13:03:02.973469  9314 tablet_bootstrap.cc:492] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:02.980824  9314 ts_tablet_manager.cc:1403] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.129s	user 0.021s	sys 0.034s
I20260709 13:03:02.981149  9182 tablet_service.cc:1511] Processing CreateTablet for tablet 9daa912f9e08438caf063c0c83e09fe5 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:02.982882  9182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9daa912f9e08438caf063c0c83e09fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:02.983620  9314 raft_consensus.cc:359] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:02.986066  9314 raft_consensus.cc:385] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:02.991673  9111 tablet_service.cc:1511] Processing CreateTablet for tablet dc3a66792b31404ba18eaacb7c54d08b (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:02.993093  9313 ts_tablet_manager.cc:1434] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.130s	user 0.015s	sys 0.011s
I20260709 13:03:02.998824  9257 tablet_service.cc:1511] Processing CreateTablet for tablet dc3a66792b31404ba18eaacb7c54d08b (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:03.000698  9148 heartbeater.cc:499] Master 127.7.213.126:45141 was elected leader, sending a full tablet report...
I20260709 13:03:02.999131  9254 tablet_service.cc:1511] Processing CreateTablet for tablet 9daa912f9e08438caf063c0c83e09fe5 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:03.001016  9257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc3a66792b31404ba18eaacb7c54d08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.004797  9254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9daa912f9e08438caf063c0c83e09fe5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:03.000370  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:03.009512  9112 tablet_service.cc:1511] Processing CreateTablet for tablet 017b65b967c44e7688e66e7aab235b62 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:02.996196  9314 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.012607  9314 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.013656  9107 tablet_service.cc:1511] Processing CreateTablet for tablet 23636a3a86df4bdf91dc6a7ffe723527 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:03.020607  9111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc3a66792b31404ba18eaacb7c54d08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.023600  9187 tablet_service.cc:1511] Processing CreateTablet for tablet dc3a66792b31404ba18eaacb7c54d08b (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:03.024645  9107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23636a3a86df4bdf91dc6a7ffe723527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.025020  9187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc3a66792b31404ba18eaacb7c54d08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.028205  9261 tablet_service.cc:1511] Processing CreateTablet for tablet c07692445f814d5cab7dd13d622f9b9c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:03.029695  9261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07692445f814d5cab7dd13d622f9b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.030313  9312 tablet_bootstrap.cc:654] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.030546  9188 tablet_service.cc:1511] Processing CreateTablet for tablet c089688bfc69403095596a002ccbf890 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:03.031978  9188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c089688bfc69403095596a002ccbf890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.033767  9260 tablet_service.cc:1511] Processing CreateTablet for tablet c089688bfc69403095596a002ccbf890 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:03.035105  9260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c089688bfc69403095596a002ccbf890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.036763  9184 tablet_service.cc:1511] Processing CreateTablet for tablet 7cfb4241815c471a8b29031ba9674528 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:03.038093  9184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cfb4241815c471a8b29031ba9674528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.037822  9180 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3c420051574414bbec6a77f8036b61 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:03.039047  9180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3c420051574414bbec6a77f8036b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.041582  9262 tablet_service.cc:1511] Processing CreateTablet for tablet 7cfb4241815c471a8b29031ba9674528 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:03.041849  9179 tablet_service.cc:1511] Processing CreateTablet for tablet a5204a9a27e54ddcb19d93142d8156f7 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:03.042982  9262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cfb4241815c471a8b29031ba9674528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.043130  9179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5204a9a27e54ddcb19d93142d8156f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.014863  9253 tablet_service.cc:1511] Processing CreateTablet for tablet 017b65b967c44e7688e66e7aab235b62 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:03.015403  9256 tablet_service.cc:1511] Processing CreateTablet for tablet 23636a3a86df4bdf91dc6a7ffe723527 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:03.044878  9255 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3c420051574414bbec6a77f8036b61 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:03.047142  9256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23636a3a86df4bdf91dc6a7ffe723527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.015528  9186 tablet_service.cc:1511] Processing CreateTablet for tablet 23636a3a86df4bdf91dc6a7ffe723527 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:03.052749  9186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23636a3a86df4bdf91dc6a7ffe723527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.016561  9313 tablet_bootstrap.cc:492] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:03.017903  9113 tablet_service.cc:1511] Processing CreateTablet for tablet c07692445f814d5cab7dd13d622f9b9c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:03.019272  9183 tablet_service.cc:1511] Processing CreateTablet for tablet 017b65b967c44e7688e66e7aab235b62 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:03.068338  9104 tablet_service.cc:1511] Processing CreateTablet for tablet 7c3c420051574414bbec6a77f8036b61 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:03.069316  9255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3c420051574414bbec6a77f8036b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.070379  9313 tablet_bootstrap.cc:654] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.072284  9183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 017b65b967c44e7688e66e7aab235b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.071197  9253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 017b65b967c44e7688e66e7aab235b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.080442  9112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 017b65b967c44e7688e66e7aab235b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.100216  9113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07692445f814d5cab7dd13d622f9b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.114763  9320 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.115291  9320 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.116533  9106 tablet_service.cc:1511] Processing CreateTablet for tablet c089688bfc69403095596a002ccbf890 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:03.118880  9297 heartbeater.cc:499] Master 127.7.213.126:45141 was elected leader, sending a full tablet report...
I20260709 13:03:03.130409  9320 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:03.131654  9185 tablet_service.cc:1511] Processing CreateTablet for tablet c07692445f814d5cab7dd13d622f9b9c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:03.133100  9185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07692445f814d5cab7dd13d622f9b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.145246  9108 tablet_service.cc:1511] Processing CreateTablet for tablet a5204a9a27e54ddcb19d93142d8156f7 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:03.145596  9109 tablet_service.cc:1511] Processing CreateTablet for tablet a6c1cc32069145ebbbacf127d2516fe8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:03.147291  9110 tablet_service.cc:1511] Processing CreateTablet for tablet 7cfb4241815c471a8b29031ba9674528 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:03.157236  9104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c3c420051574414bbec6a77f8036b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.164110  9106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c089688bfc69403095596a002ccbf890. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.174101  9314 ts_tablet_manager.cc:1434] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.193s	user 0.011s	sys 0.025s
I20260709 13:03:03.188367  9181 tablet_service.cc:1511] Processing CreateTablet for tablet a6c1cc32069145ebbbacf127d2516fe8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:03.189879  9181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c1cc32069145ebbbacf127d2516fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.197337  9314 tablet_bootstrap.cc:492] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:03.203375  9110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cfb4241815c471a8b29031ba9674528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.229478  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:03.232028  9235 leader_election.cc:343] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:03.240900  9107 tablet_service.cc:1511] Processing CreateTablet for tablet 00ff82ce7b7342a4b052db43182f4a99 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:03.252954  9112 tablet_service.cc:1511] Processing CreateTablet for tablet b0917335c1234d5abec1e40c742de6be (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:03.254622  9112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0917335c1234d5abec1e40c742de6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.263355  9109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c1cc32069145ebbbacf127d2516fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.251394  9181 tablet_service.cc:1511] Processing CreateTablet for tablet 00ff82ce7b7342a4b052db43182f4a99 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:03.265236  9181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00ff82ce7b7342a4b052db43182f4a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.280599  9111 tablet_service.cc:1511] Processing CreateTablet for tablet 470326df6cae476c90888533e504e6ec (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:03.288532  9105 tablet_service.cc:1511] Processing CreateTablet for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:03.290634  9108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5204a9a27e54ddcb19d93142d8156f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.293033  9105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.305125  9253 tablet_service.cc:1511] Processing CreateTablet for tablet 00ff82ce7b7342a4b052db43182f4a99 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:03.306505  9253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00ff82ce7b7342a4b052db43182f4a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.306999  9312 tablet_bootstrap.cc:492] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:03.307636  9312 ts_tablet_manager.cc:1403] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.334s	user 0.030s	sys 0.034s
I20260709 13:03:03.307564  9112 tablet_service.cc:1511] Processing CreateTablet for tablet bc89bbd2f2934bbeaf33619d3d23adfc (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:03.308751  9254 tablet_service.cc:1511] Processing CreateTablet for tablet 470326df6cae476c90888533e504e6ec (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:03.308992  9256 tablet_service.cc:1511] Processing CreateTablet for tablet b0917335c1234d5abec1e40c742de6be (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:03.310019  9254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470326df6cae476c90888533e504e6ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.311864  9262 tablet_service.cc:1511] Processing CreateTablet for tablet a6c1cc32069145ebbbacf127d2516fe8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:03.312753  9257 tablet_service.cc:1511] Processing CreateTablet for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:03.313174  9255 tablet_service.cc:1511] Processing CreateTablet for tablet a5204a9a27e54ddcb19d93142d8156f7 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:03.315673  9257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.302502  9107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00ff82ce7b7342a4b052db43182f4a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.324023  9255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5204a9a27e54ddcb19d93142d8156f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.327332  9187 tablet_service.cc:1511] Processing CreateTablet for tablet 470326df6cae476c90888533e504e6ec (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:03.348740  9187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470326df6cae476c90888533e504e6ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.328433  9313 tablet_bootstrap.cc:492] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:03.350950  9313 ts_tablet_manager.cc:1403] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.335s	user 0.022s	sys 0.038s
I20260709 13:03:03.350890  9318 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.351516  9318 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.334610  9188 tablet_service.cc:1511] Processing CreateTablet for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:03.350930  9256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0917335c1234d5abec1e40c742de6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.360301  9314 tablet_bootstrap.cc:654] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.298513  9184 tablet_service.cc:1511] Processing CreateTablet for tablet b0917335c1234d5abec1e40c742de6be (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:03.364795  9312 raft_consensus.cc:359] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.365607  9312 raft_consensus.cc:385] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.365917  9312 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.367259  9188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.368067  9184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0917335c1234d5abec1e40c742de6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.368286  9112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc89bbd2f2934bbeaf33619d3d23adfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.376331  9261 tablet_service.cc:1511] Processing CreateTablet for tablet bc89bbd2f2934bbeaf33619d3d23adfc (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:03.377673  9312 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.380142  9318 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:03.385156  9260 tablet_service.cc:1511] Processing CreateTablet for tablet 2b7da62e1fea4b79963197feee655d26 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:03.390599  9111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470326df6cae476c90888533e504e6ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.403791  9319 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.397136  9113 tablet_service.cc:1511] Processing CreateTablet for tablet 8a4bd074bf594b89b28b47d7482e8ac1 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:03.404470  9319 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.405674  9113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4bd074bf594b89b28b47d7482e8ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.414806  9106 tablet_service.cc:1511] Processing CreateTablet for tablet 2b7da62e1fea4b79963197feee655d26 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:03.416465  9186 tablet_service.cc:1511] Processing CreateTablet for tablet bc89bbd2f2934bbeaf33619d3d23adfc (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:03.417979  9186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc89bbd2f2934bbeaf33619d3d23adfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.420626  9106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b7da62e1fea4b79963197feee655d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.434506  9313 raft_consensus.cc:359] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.435276  9313 raft_consensus.cc:385] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.435560  9313 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.444978  9319 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:03.449317  9262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c1cc32069145ebbbacf127d2516fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.462117  9108 tablet_service.cc:1511] Processing CreateTablet for tablet 9e2655d16bd349309dab52831e66fee8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:03.463588  9108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2655d16bd349309dab52831e66fee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.480412  9260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b7da62e1fea4b79963197feee655d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.456915  9313 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.496860  9261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc89bbd2f2934bbeaf33619d3d23adfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.498124  9329 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.498627  9329 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.505371  9329 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:03.508399  9104 tablet_service.cc:1511] Processing CreateTablet for tablet db3bbfae79ea4f4cb8b7e3079bdb3908 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:03.509773  9104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3bbfae79ea4f4cb8b7e3079bdb3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.516973  9313 ts_tablet_manager.cc:1434] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.166s	user 0.003s	sys 0.004s
I20260709 13:03:03.518050  9313 tablet_bootstrap.cc:492] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:03.520314  9109 tablet_service.cc:1511] Processing CreateTablet for tablet 1abbfa6c01a144acb9bc31470b44be7e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:03.521685  9109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1abbfa6c01a144acb9bc31470b44be7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.527082  9107 tablet_service.cc:1511] Processing CreateTablet for tablet edfac5554fb144c3ab8a6248c7ea68d8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:03.528522  9254 tablet_service.cc:1511] Processing CreateTablet for tablet 8a4bd074bf594b89b28b47d7482e8ac1 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:03.551143  9254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4bd074bf594b89b28b47d7482e8ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.531984  9108 tablet_service.cc:1511] Processing CreateTablet for tablet 909051b50b2c464ea2b13a15a243817c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:03.552398  9112 tablet_service.cc:1511] Processing CreateTablet for tablet c351454182cf49938eb92870b1fb297e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:03.553143  9108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909051b50b2c464ea2b13a15a243817c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.554677  9110 tablet_service.cc:1511] Processing CreateTablet for tablet 29257e9d0bc4408583c5df86d6e2fd64 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:03.556720  9185 tablet_service.cc:1511] Processing CreateTablet for tablet 8a4bd074bf594b89b28b47d7482e8ac1 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:03.558168  9185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4bd074bf594b89b28b47d7482e8ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.536006  9312 ts_tablet_manager.cc:1434] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.228s	user 0.015s	sys 0.009s
I20260709 13:03:03.556941  9184 tablet_service.cc:1511] Processing CreateTablet for tablet 1abbfa6c01a144acb9bc31470b44be7e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:03.560709  9184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1abbfa6c01a144acb9bc31470b44be7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.564163  9110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29257e9d0bc4408583c5df86d6e2fd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.563758  9179 tablet_service.cc:1511] Processing CreateTablet for tablet 2b7da62e1fea4b79963197feee655d26 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:03.565398  9179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b7da62e1fea4b79963197feee655d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.568097  9112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c351454182cf49938eb92870b1fb297e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.569998  9312 tablet_bootstrap.cc:492] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:03.581068  9183 tablet_service.cc:1511] Processing CreateTablet for tablet 9e2655d16bd349309dab52831e66fee8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:03.582367  9183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2655d16bd349309dab52831e66fee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.585229  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:03.586396  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:03.590113  9188 tablet_service.cc:1511] Processing CreateTablet for tablet edfac5554fb144c3ab8a6248c7ea68d8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:03.600531  9188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edfac5554fb144c3ab8a6248c7ea68d8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:03.600275  9162 leader_election.cc:343] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:03.601926  9162 leader_election.cc:343] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:03.603705  9312 tablet_bootstrap.cc:654] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.598429  9107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edfac5554fb144c3ab8a6248c7ea68d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.612684  9180 tablet_service.cc:1511] Processing CreateTablet for tablet 909051b50b2c464ea2b13a15a243817c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:03.614567  9180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909051b50b2c464ea2b13a15a243817c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.621908  9313 tablet_bootstrap.cc:654] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:03.614087  9186 tablet_service.cc:1511] Processing CreateTablet for tablet db3bbfae79ea4f4cb8b7e3079bdb3908 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:03.655936  9105 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdaa422821a46b8bc73801d1abfb3c0 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:03.657464  9105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdaa422821a46b8bc73801d1abfb3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.679368  9186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3bbfae79ea4f4cb8b7e3079bdb3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.698021  9108 tablet_service.cc:1511] Processing CreateTablet for tablet bbfd5d6417134672a62b2757b6d55dff (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:03.697059  9113 tablet_service.cc:1511] Processing CreateTablet for tablet a08146adfeb9417ca7067de38da555b0 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:03.699613  9108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfd5d6417134672a62b2757b6d55dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.705713  9111 tablet_service.cc:1511] Processing CreateTablet for tablet 0a06199955d24d13ba9589faa3f11ebe (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:03.707078  9111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a06199955d24d13ba9589faa3f11ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.711184  9262 tablet_service.cc:1511] Processing CreateTablet for tablet 1abbfa6c01a144acb9bc31470b44be7e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:03.711328  9182 tablet_service.cc:1511] Processing CreateTablet for tablet 29257e9d0bc4408583c5df86d6e2fd64 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:03.712731  9262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1abbfa6c01a144acb9bc31470b44be7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.713101  9182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29257e9d0bc4408583c5df86d6e2fd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.676765  9183 tablet_service.cc:1511] Processing CreateTablet for tablet c351454182cf49938eb92870b1fb297e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:03.675418  9106 tablet_service.cc:1511] Processing CreateTablet for tablet 9a3eaf34ab444dca94a52a6fce2a556e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:03.735272  9314 tablet_bootstrap.cc:492] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:03.735842  9314 ts_tablet_manager.cc:1403] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.559s	user 0.021s	sys 0.013s
I20260709 13:03:03.739095  9314 raft_consensus.cc:359] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.740005  9314 raft_consensus.cc:385] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.740356  9314 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.740849  9183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c351454182cf49938eb92870b1fb297e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.741096  9314 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.726410  9110 tablet_service.cc:1511] Processing CreateTablet for tablet aaf63fb358d848ad991a929b2e8e0168 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:03.746228  9256 tablet_service.cc:1511] Processing CreateTablet for tablet 9e2655d16bd349309dab52831e66fee8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:03.749396  9261 tablet_service.cc:1511] Processing CreateTablet for tablet db3bbfae79ea4f4cb8b7e3079bdb3908 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:03.751362  9261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3bbfae79ea4f4cb8b7e3079bdb3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.754344  9181 tablet_service.cc:1511] Processing CreateTablet for tablet 9a3eaf34ab444dca94a52a6fce2a556e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:03.755831  9181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a3eaf34ab444dca94a52a6fce2a556e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.762725  9185 tablet_service.cc:1511] Processing CreateTablet for tablet a08146adfeb9417ca7067de38da555b0 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:03.730460  9113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a08146adfeb9417ca7067de38da555b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.768175  9106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a3eaf34ab444dca94a52a6fce2a556e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.764144  9185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a08146adfeb9417ca7067de38da555b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.792959  9260 tablet_service.cc:1511] Processing CreateTablet for tablet edfac5554fb144c3ab8a6248c7ea68d8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:03.793777  9255 tablet_service.cc:1511] Processing CreateTablet for tablet c351454182cf49938eb92870b1fb297e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:03.793905  9257 tablet_service.cc:1511] Processing CreateTablet for tablet 909051b50b2c464ea2b13a15a243817c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:03.792932  9105 tablet_service.cc:1511] Processing CreateTablet for tablet a35f9598c50046539a7f41287876135d (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:03.794071  9253 tablet_service.cc:1511] Processing CreateTablet for tablet 29257e9d0bc4408583c5df86d6e2fd64 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:03.795692  9256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2655d16bd349309dab52831e66fee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.796707  9255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c351454182cf49938eb92870b1fb297e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.797298  9260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edfac5554fb144c3ab8a6248c7ea68d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.797422  9258 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdaa422821a46b8bc73801d1abfb3c0 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:03.798713  9258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdaa422821a46b8bc73801d1abfb3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.798571  9182 tablet_service.cc:1511] Processing CreateTablet for tablet 0a06199955d24d13ba9589faa3f11ebe (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:03.799798  9182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a06199955d24d13ba9589faa3f11ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.800021  9257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909051b50b2c464ea2b13a15a243817c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.804078  9253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29257e9d0bc4408583c5df86d6e2fd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.804971  9184 tablet_service.cc:1511] Processing CreateTablet for tablet aaf63fb358d848ad991a929b2e8e0168 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:03.810322  9184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf63fb358d848ad991a929b2e8e0168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.805848  9110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf63fb358d848ad991a929b2e8e0168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.812502  9179 tablet_service.cc:1511] Processing CreateTablet for tablet 1fdaa422821a46b8bc73801d1abfb3c0 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:03.813773  9179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdaa422821a46b8bc73801d1abfb3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.823984  9259 tablet_service.cc:1511] Processing CreateTablet for tablet a08146adfeb9417ca7067de38da555b0 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:03.832615  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:03.837821  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:03.838495  9198 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:03.840076  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:03.842104  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:03.842743  9123 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:03.844048  9319 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.844523  9319 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.846710  9319 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:03.848575  9089 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:03.849540  9319 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:03.849902  9319 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.850248  9319 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:03.856748  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:03.859465  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:03.860193  9105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a35f9598c50046539a7f41287876135d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:03.877097  9238 leader_election.cc:343] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:03.877672  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:03.855170  9254 tablet_service.cc:1511] Processing CreateTablet for tablet aaf63fb358d848ad991a929b2e8e0168 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 13:03:03.858021  9161 leader_election.cc:343] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:03.861685  9313 tablet_bootstrap.cc:492] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:03.861984  9314 ts_tablet_manager.cc:1434] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.126s	user 0.028s	sys 0.031s
I20260709 13:03:03.871709  9257 tablet_service.cc:1511] Processing CreateTablet for tablet 9a3eaf34ab444dca94a52a6fce2a556e (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
W20260709 13:03:03.875597  9087 leader_election.cc:343] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:03.859716  9180 tablet_service.cc:1511] Processing CreateTablet for tablet bbfd5d6417134672a62b2757b6d55dff (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:03.888764  9259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a08146adfeb9417ca7067de38da555b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.889978  9313 ts_tablet_manager.cc:1403] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.372s	user 0.047s	sys 0.038s
I20260709 13:03:03.902846  9313 raft_consensus.cc:359] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.903630  9313 raft_consensus.cc:385] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:03.904233  9313 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:03.905099  9313 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.908311  9254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf63fb358d848ad991a929b2e8e0168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.873458  9107 tablet_service.cc:1511] Processing CreateTablet for tablet 55b71b09f6834fbbbd500874ff1e14b5 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:03.912830  9107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b71b09f6834fbbbd500874ff1e14b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.912647  9260 tablet_service.cc:1511] Processing CreateTablet for tablet bbfd5d6417134672a62b2757b6d55dff (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:03.913908  9260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfd5d6417134672a62b2757b6d55dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.897326  9112 tablet_service.cc:1511] Processing CreateTablet for tablet 2cfe3483aaf54fbd932e71b72394491c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:03.917418  9112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cfe3483aaf54fbd932e71b72394491c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.925493  9336 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:03.926690  9108 tablet_service.cc:1511] Processing CreateTablet for tablet e9a6ffaa328644f4bd7f129f8fa22875 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:03.933408  9319 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:03.936309  9314 tablet_bootstrap.cc:492] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:03.936116  9253 tablet_service.cc:1511] Processing CreateTablet for tablet 0a06199955d24d13ba9589faa3f11ebe (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:03.965656  9108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a6ffaa328644f4bd7f129f8fa22875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.967540  9312 tablet_bootstrap.cc:492] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:03.968233  9312 ts_tablet_manager.cc:1403] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.399s	user 0.021s	sys 0.003s
I20260709 13:03:03.890518  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:03.969985  9329 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:03.970180  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 112312us
I20260709 13:03:03.974184  9180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfd5d6417134672a62b2757b6d55dff. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:03.973976  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 97059.5us
I20260709 13:03:03.976111  9257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a3eaf34ab444dca94a52a6fce2a556e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:03.976307  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:03.977144  9272 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:03.977473  9253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a06199955d24d13ba9589faa3f11ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.002552  9313 ts_tablet_manager.cc:1434] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.102s	user 0.012s	sys 0.008s
I20260709 13:03:04.017715  9319 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:03:04.018591  9005 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 51744.8us
W20260709 13:03:04.019654  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:04.022132  9312 raft_consensus.cc:359] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.026602  9312 raft_consensus.cc:385] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.027086  9312 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.029181  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:04.033485  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:04.046934  9312 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.003657  9313 tablet_bootstrap.cc:492] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
W20260709 13:03:04.048352  9087 leader_election.cc:343] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.050700  9113 tablet_service.cc:1511] Processing CreateTablet for tablet 9cbe8a4b1d764261adeeb8d882fa37e5 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:04.052103  9113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cbe8a4b1d764261adeeb8d882fa37e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.048635  9319 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.054232  9319 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.055073  9319 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.053804  9312 ts_tablet_manager.cc:1434] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.085s	user 0.010s	sys 0.006s
I20260709 13:03:04.043107  9258 tablet_service.cc:1511] Processing CreateTablet for tablet a35f9598c50046539a7f41287876135d (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:04.040076  9161 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:04.057781  9256 tablet_service.cc:1511] Processing CreateTablet for tablet 55b71b09f6834fbbbd500874ff1e14b5 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:04.058451  9258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a35f9598c50046539a7f41287876135d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.059244  9256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b71b09f6834fbbbd500874ff1e14b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.061194  9329 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.061650  9329 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.062062  9329 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.075263  9312 tablet_bootstrap.cc:492] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
W20260709 13:03:04.127171  9006 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 75077.8us
I20260709 13:03:04.137717  9314 tablet_bootstrap.cc:654] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.140223  9313 tablet_bootstrap.cc:654] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.143618  9329 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:04.144045  9005 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 87890us
I20260709 13:03:04.150490  9312 tablet_bootstrap.cc:654] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.152029  9106 tablet_service.cc:1511] Processing CreateTablet for tablet 6cdcabf0b1ec4b319a537ed3662ce253 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:04.153285  9106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cdcabf0b1ec4b319a537ed3662ce253. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:04.159771  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 107947us
W20260709 13:03:04.162039  9089 leader_election.cc:343] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.165032  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:03:04.167225  9162 leader_election.cc:343] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.148518  9260 tablet_service.cc:1511] Processing CreateTablet for tablet e9a6ffaa328644f4bd7f129f8fa22875 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:04.168560  9260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a6ffaa328644f4bd7f129f8fa22875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.146515  9319 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.172106  9314 tablet_bootstrap.cc:492] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:04.172607  9314 ts_tablet_manager.cc:1403] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.237s	user 0.020s	sys 0.018s
I20260709 13:03:04.173621  9342 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.173763  9111 tablet_service.cc:1511] Processing CreateTablet for tablet 06c3356087cc45e78fa0f79168b68e8c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:04.174101  9342 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.175091  9111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c3356087cc45e78fa0f79168b68e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.176137  9342 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.177336  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:04.179589  9110 tablet_service.cc:1511] Processing CreateTablet for tablet 90c9bcfd54594ec58c5e8e1be6c556d8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:04.180881  9110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90c9bcfd54594ec58c5e8e1be6c556d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.182893  9256 tablet_service.cc:1511] Processing CreateTablet for tablet 2cfe3483aaf54fbd932e71b72394491c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:04.184324  9256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cfe3483aaf54fbd932e71b72394491c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.192659  9255 tablet_service.cc:1511] Processing CreateTablet for tablet 9cbe8a4b1d764261adeeb8d882fa37e5 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:04.194726  9105 tablet_service.cc:1511] Processing CreateTablet for tablet aa32d6e0bb7b44ee9f664126e3ecdb72 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:04.192689  9253 tablet_service.cc:1511] Processing CreateTablet for tablet aa32d6e0bb7b44ee9f664126e3ecdb72 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:04.196914  9255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cbe8a4b1d764261adeeb8d882fa37e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.202092  9253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa32d6e0bb7b44ee9f664126e3ecdb72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.204566  9319 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.205618  9105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa32d6e0bb7b44ee9f664126e3ecdb72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.205783  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:04.206380  9271 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.208252  9181 tablet_service.cc:1511] Processing CreateTablet for tablet aa32d6e0bb7b44ee9f664126e3ecdb72 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:04.208732  9179 tablet_service.cc:1511] Processing CreateTablet for tablet e9a6ffaa328644f4bd7f129f8fa22875 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:04.175616  9314 raft_consensus.cc:359] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.209553  9181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa32d6e0bb7b44ee9f664126e3ecdb72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.216665  9184 tablet_service.cc:1511] Processing CreateTablet for tablet 9cbe8a4b1d764261adeeb8d882fa37e5 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:04.217070  9182 tablet_service.cc:1511] Processing CreateTablet for tablet 6cdcabf0b1ec4b319a537ed3662ce253 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:04.218094  9183 tablet_service.cc:1511] Processing CreateTablet for tablet 55b71b09f6834fbbbd500874ff1e14b5 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:04.207322  9188 tablet_service.cc:1511] Processing CreateTablet for tablet 2cfe3483aaf54fbd932e71b72394491c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:04.211190  9314 raft_consensus.cc:385] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.220739  9185 tablet_service.cc:1511] Processing CreateTablet for tablet a35f9598c50046539a7f41287876135d (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:04.221822  9182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cdcabf0b1ec4b319a537ed3662ce253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.225049  9185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a35f9598c50046539a7f41287876135d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.225900  9179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a6ffaa328644f4bd7f129f8fa22875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.228057  9184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cbe8a4b1d764261adeeb8d882fa37e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.232079  9183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b71b09f6834fbbbd500874ff1e14b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.232749  9188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cfe3483aaf54fbd932e71b72394491c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.196835  9107 tablet_service.cc:1511] Processing CreateTablet for tablet 709565930ea044f08322d94e29d83adf (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:04.232486  9314 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.233947  9107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709565930ea044f08322d94e29d83adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.188763  9329 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.240862  9113 tablet_service.cc:1511] Processing CreateTablet for tablet dfa5110f517a4ffb866953379f5d0c4d (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:04.247109  9187 tablet_service.cc:1511] Processing CreateTablet for tablet 06c3356087cc45e78fa0f79168b68e8c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:04.247655  9113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfa5110f517a4ffb866953379f5d0c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.248701  9187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c3356087cc45e78fa0f79168b68e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.249585  9261 tablet_service.cc:1511] Processing CreateTablet for tablet 709565930ea044f08322d94e29d83adf (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:04.251081  9261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709565930ea044f08322d94e29d83adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.261039  9312 tablet_bootstrap.cc:492] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.261729  9312 ts_tablet_manager.cc:1403] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.187s	user 0.025s	sys 0.030s
I20260709 13:03:04.262185  9186 tablet_service.cc:1511] Processing CreateTablet for tablet 90c9bcfd54594ec58c5e8e1be6c556d8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:04.264230  9186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90c9bcfd54594ec58c5e8e1be6c556d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.273823  9254 tablet_service.cc:1511] Processing CreateTablet for tablet 06c3356087cc45e78fa0f79168b68e8c (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:04.275285  9254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c3356087cc45e78fa0f79168b68e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.290913  9338 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.291465  9338 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.299207  9262 tablet_service.cc:1511] Processing CreateTablet for tablet 90c9bcfd54594ec58c5e8e1be6c556d8 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:04.304015  9314 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.304967  9313 tablet_bootstrap.cc:492] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:04.305089  9272 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:04.305459  9313 ts_tablet_manager.cc:1403] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.302s	user 0.011s	sys 0.011s
I20260709 13:03:04.306409  9314 ts_tablet_manager.cc:1434] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.134s	user 0.007s	sys 0.000s
I20260709 13:03:04.307453  9314 tablet_bootstrap.cc:492] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:04.311921  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:04.312769  9329 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.313127  9329 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.313495  9329 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.314963  9312 raft_consensus.cc:359] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.315709  9312 raft_consensus.cc:385] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.316010  9312 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.316861  9271 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:04.316774  9312 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.318320  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:04.319090  9319 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:04.320228  9319 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:04.321074  9319 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.325567  9338 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:04.325779  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:04.325920  9110 tablet_service.cc:1511] Processing CreateTablet for tablet 49752620cbbf430991915a2400f71286 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:04.327158  9110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49752620cbbf430991915a2400f71286. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:04.327427  9235 leader_election.cc:343] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.333169  9312 ts_tablet_manager.cc:1434] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.005s
I20260709 13:03:04.334293  9312 tablet_bootstrap.cc:492] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:04.334556  9329 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.341251  9180 tablet_service.cc:1511] Processing CreateTablet for tablet dfa5110f517a4ffb866953379f5d0c4d (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:04.342602  9180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfa5110f517a4ffb866953379f5d0c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.343021  9258 tablet_service.cc:1511] Processing CreateTablet for tablet dfa5110f517a4ffb866953379f5d0c4d (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:04.345001  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:04.345798  9198 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:04.348871  9257 tablet_service.cc:1511] Processing CreateTablet for tablet 49752620cbbf430991915a2400f71286 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:04.351055  9108 tablet_service.cc:1511] Processing CreateTablet for tablet c0f89ac8f7b142239d6103bd6ed96037 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:04.352465  9108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f89ac8f7b142239d6103bd6ed96037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.354163  9339 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.354674  9339 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.291472  9259 tablet_service.cc:1511] Processing CreateTablet for tablet 6cdcabf0b1ec4b319a537ed3662ce253 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:04.356997  9339 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.358325  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:04.359803  9087 leader_election.cc:343] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.368269  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:04.369697  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:04.370170  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:04.370533  9123 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:04.370639  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:04.371475  9238 leader_election.cc:343] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.371573  9089 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:04.372130  9238 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:04.371549  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:04.373095  9196 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:04.373162  9339 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:04.371652  9198 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.373692  9161 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:04.374540  9342 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:03:04.375703  9161 leader_election.cc:343] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.377844  9262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90c9bcfd54594ec58c5e8e1be6c556d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.373634  9338 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:04.385236  9258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfa5110f517a4ffb866953379f5d0c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.385265  9089 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:04.387966  9198 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:04.389266  9259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cdcabf0b1ec4b319a537ed3662ce253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.389981  9329 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.391491  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:04.392275  9271 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.391935  9019 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.398531  9313 raft_consensus.cc:359] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.407948  9313 raft_consensus.cc:385] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.408339  9313 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.409245  9313 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.410938  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:03:04.413030  9161 leader_election.cc:343] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.417433  9185 tablet_service.cc:1511] Processing CreateTablet for tablet c0f89ac8f7b142239d6103bd6ed96037 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:04.412822  9260 tablet_service.cc:1511] Processing CreateTablet for tablet c0f89ac8f7b142239d6103bd6ed96037 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:04.418692  9185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f89ac8f7b142239d6103bd6ed96037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.430948  9271 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:04.413367  9313 ts_tablet_manager.cc:1434] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.108s	user 0.006s	sys 0.002s
I20260709 13:03:04.397786  9319 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.432185  9319 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.432337  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:04.432626  9319 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.433624  9329 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:04.434907  9329 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:04.410959  9257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49752620cbbf430991915a2400f71286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.435943  9329 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.440343  9187 tablet_service.cc:1511] Processing CreateTablet for tablet 49752620cbbf430991915a2400f71286 (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:04.441768  9187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49752620cbbf430991915a2400f71286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.445935  9314 tablet_bootstrap.cc:654] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.448621  9313 tablet_bootstrap.cc:492] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:04.449576  9319 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.425223  9312 tablet_bootstrap.cc:654] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.476524  9319 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.456954  9260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f89ac8f7b142239d6103bd6ed96037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.478926  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:04.403575  9179 tablet_service.cc:1511] Processing CreateTablet for tablet 709565930ea044f08322d94e29d83adf (DEFAULT_TABLE table=test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
W20260709 13:03:04.480232  9087 leader_election.cc:343] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.486735  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:04.487349  9198 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.488379  9314 tablet_bootstrap.cc:492] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:04.488881  9314 ts_tablet_manager.cc:1403] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.182s	user 0.012s	sys 0.005s
I20260709 13:03:04.491138  9312 tablet_bootstrap.cc:492] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.491811  9312 ts_tablet_manager.cc:1403] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.158s	user 0.011s	sys 0.002s
I20260709 13:03:04.493742  9179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709565930ea044f08322d94e29d83adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:04.497470  9312 raft_consensus.cc:359] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.498258  9312 raft_consensus.cc:385] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.498591  9312 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.493425  9314 raft_consensus.cc:359] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.497951  9198 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:04.499951  9313 tablet_bootstrap.cc:654] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.508104  9089 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:04.509284  9319 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:04.509786  9319 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:04.499336  9312 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.502033  9020 catalog_manager.cc:5697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.510805  9319 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.512514  9312 ts_tablet_manager.cc:1434] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 13:03:04.513438  9339 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.506385  9314 raft_consensus.cc:385] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.513945  9339 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.520136  9339 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.522001  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:04.522387  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:04.523058  9271 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:04.524282  9087 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:04.514272  9314 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.525094  9353 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.525481  9353 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.525871  9353 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.525563  9314 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.527940  9314 ts_tablet_manager.cc:1434] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:03:04.529004  9314 tablet_bootstrap.cc:492] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:04.533325  9312 tablet_bootstrap.cc:492] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:04.534317  9353 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:04.536158  9089 leader_election.cc:343] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.537936  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:04.538641  9271 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.538561  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:04.546489  9314 tablet_bootstrap.cc:654] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.546895  9353 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.549427  9313 tablet_bootstrap.cc:492] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:04.549954  9313 ts_tablet_manager.cc:1403] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.102s	user 0.006s	sys 0.010s
I20260709 13:03:04.552907  9271 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:04.553210  9313 raft_consensus.cc:359] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.553954  9313 raft_consensus.cc:385] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.554304  9313 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.555044  9313 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.557540  9313 ts_tablet_manager.cc:1434] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:04.558620  9313 tablet_bootstrap.cc:492] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:04.560115  9312 tablet_bootstrap.cc:654] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.565479  9313 tablet_bootstrap.cc:654] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:04.553668  9089 leader_election.cc:343] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.575085  9087 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:04.576555  9353 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:04.577212  9353 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:04.578282  9353 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.597851  9312 tablet_bootstrap.cc:492] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.598373  9312 ts_tablet_manager.cc:1403] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.010s
I20260709 13:03:04.598471  9313 tablet_bootstrap.cc:492] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:04.599013  9313 ts_tablet_manager.cc:1403] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.041s	user 0.006s	sys 0.020s
I20260709 13:03:04.600433  9314 tablet_bootstrap.cc:492] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:04.601023  9314 ts_tablet_manager.cc:1403] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.016s
I20260709 13:03:04.600611  9019 catalog_manager.cc:5697] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.601586  9312 raft_consensus.cc:359] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.602311  9312 raft_consensus.cc:385] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.602085  9313 raft_consensus.cc:359] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.602654  9312 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.602847  9313 raft_consensus.cc:385] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.603336  9313 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.603416  9312 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.604156  9313 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.606324  9313 ts_tablet_manager.cc:1434] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:03:04.607307  9313 tablet_bootstrap.cc:492] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:04.606863  9314 raft_consensus.cc:359] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.609869  9314 raft_consensus.cc:385] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.610203  9314 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.611369  9312 ts_tablet_manager.cc:1434] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:03:04.612407  9312 tablet_bootstrap.cc:492] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:04.614059  9313 tablet_bootstrap.cc:654] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.618834  9312 tablet_bootstrap.cc:654] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.622413  9018 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.623502  9312 tablet_bootstrap.cc:492] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.624191  9312 ts_tablet_manager.cc:1403] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:03:04.624404  9314 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.625844  9313 tablet_bootstrap.cc:492] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:04.626370  9313 ts_tablet_manager.cc:1403] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.009s
I20260709 13:03:04.626755  9314 ts_tablet_manager.cc:1434] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.025s	user 0.001s	sys 0.003s
I20260709 13:03:04.628172  9314 tablet_bootstrap.cc:492] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:04.629400  9313 raft_consensus.cc:359] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.630178  9313 raft_consensus.cc:385] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.630519  9313 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.629997  9312 raft_consensus.cc:359] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.631418  9312 raft_consensus.cc:385] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.631251  9313 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.631784  9312 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.632611  9312 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.633989  9313 ts_tablet_manager.cc:1434] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:04.634996  9313 tablet_bootstrap.cc:492] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:04.640412  9312 ts_tablet_manager.cc:1434] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:03:04.641641  9312 tablet_bootstrap.cc:492] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:04.641752  9313 tablet_bootstrap.cc:654] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.645247  9338 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.645799  9338 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.647902  9338 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:04.650269  9314 tablet_bootstrap.cc:654] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.650430  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:04.651078  9122 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:04.652107  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:04.652757  9198 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:04.654397  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:04.655295  9312 tablet_bootstrap.cc:654] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.657022  9338 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.657616  9338 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.658053  9338 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.670835  9312 tablet_bootstrap.cc:492] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.671334  9312 ts_tablet_manager.cc:1403] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.008s
I20260709 13:03:04.674456  9312 raft_consensus.cc:359] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.675429  9312 raft_consensus.cc:385] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.675863  9312 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.677000  9312 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.677590  9338 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.682111  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:04.682994  9122 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.685487  9312 ts_tablet_manager.cc:1434] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:03:04.686841  9312 tablet_bootstrap.cc:492] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:04.695073  9312 tablet_bootstrap.cc:654] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.708211  9312 tablet_bootstrap.cc:492] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.708837  9312 ts_tablet_manager.cc:1403] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 13:03:04.712818  9338 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:04.714253  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:04.714802  9198 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.716248  9313 tablet_bootstrap.cc:492] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:04.716157  9122 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:04.717003  9313 ts_tablet_manager.cc:1403] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.082s	user 0.032s	sys 0.045s
I20260709 13:03:04.717598  9314 tablet_bootstrap.cc:492] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:04.717554  9235 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:04.718152  9314 ts_tablet_manager.cc:1403] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.090s	user 0.047s	sys 0.024s
I20260709 13:03:04.718248  9319 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:04.718554  9338 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:04.719853  9338 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:04.720007  9313 raft_consensus.cc:359] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.720808  9313 raft_consensus.cc:385] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.720726  9338 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.721238  9313 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.721112  9314 raft_consensus.cc:359] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.721894  9314 raft_consensus.cc:385] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.722231  9314 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.722358  9313 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.724797  9313 ts_tablet_manager.cc:1434] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:04.723052  9314 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.726444  9313 tablet_bootstrap.cc:492] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:04.729565  9198 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:04.730507  9314 ts_tablet_manager.cc:1434] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:03:04.731602  9314 tablet_bootstrap.cc:492] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:04.712375  9312 raft_consensus.cc:359] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.734172  9313 tablet_bootstrap.cc:654] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.741593  9319 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:04.745507  9312 raft_consensus.cc:385] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.745846  9312 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.746510  9312 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.748602  9312 ts_tablet_manager.cc:1434] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:03:04.750494  9336 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.750799  9312 tablet_bootstrap.cc:492] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:04.750979  9336 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.754652  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:04.755825  9196 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:04.759441  9312 tablet_bootstrap.cc:654] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.761791  9319 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.763020  9319 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.760497  9336 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:04.765204  9319 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.765540  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:04.767452  9235 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:04.769311  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:04.769972  9272 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:04.771015  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:04.771670  9196 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:04.774384  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:04.774722  9314 tablet_bootstrap.cc:654] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.775341  9319 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.775722  9319 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.772552  9336 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.776216  9319 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.777585  9021 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:04.788327  9313 tablet_bootstrap.cc:492] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:04.788817  9313 ts_tablet_manager.cc:1403] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.063s	user 0.007s	sys 0.003s
I20260709 13:03:04.785739  9319 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.790706  9329 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.791277  9329 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.792176  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:04.792769  9271 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.793414  9329 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.791493  9313 raft_consensus.cc:359] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.794327  9313 raft_consensus.cc:385] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.794687  9313 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.794932  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:04.791002  9319 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.795670  9313 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.796761  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:04.795797  9272 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:04.793267  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:04.798555  9196 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.799176  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:04.801728  9329 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.802748  9313 ts_tablet_manager.cc:1434] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:04.803998  9313 tablet_bootstrap.cc:492] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:04.806223  9271 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:04.807598  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:04.808668  9353 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:04.809167  9353 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:04.809964  9353 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.811414  9336 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:04.815899  9312 tablet_bootstrap.cc:492] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.816545  9312 ts_tablet_manager.cc:1403] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.066s	user 0.015s	sys 0.008s
I20260709 13:03:04.822710  9314 tablet_bootstrap.cc:492] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:04.823304  9314 ts_tablet_manager.cc:1403] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.092s	user 0.020s	sys 0.011s
I20260709 13:03:04.824724  9313 tablet_bootstrap.cc:654] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.826393  9314 raft_consensus.cc:359] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.827212  9314 raft_consensus.cc:385] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.827620  9314 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.827666  9312 raft_consensus.cc:359] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.828356  9329 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:04.828421  9312 raft_consensus.cc:385] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.828925  9312 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.828807  9314 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.829761  9312 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.831561  9314 ts_tablet_manager.cc:1434] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:03:04.832858  9314 tablet_bootstrap.cc:492] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:04.831656  9312 ts_tablet_manager.cc:1434] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:03:04.836570  9312 tablet_bootstrap.cc:492] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:04.837206  9196 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:04.841233  9314 tablet_bootstrap.cc:654] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.842335  9313 tablet_bootstrap.cc:492] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:04.842926  9313 ts_tablet_manager.cc:1403] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.039s	user 0.008s	sys 0.010s
I20260709 13:03:04.843518  9312 tablet_bootstrap.cc:654] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.846202  9313 raft_consensus.cc:359] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.847126  9313 raft_consensus.cc:385] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.847463  9313 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.841562  9020 catalog_manager.cc:5697] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.848541  9313 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.851657  9313 ts_tablet_manager.cc:1434] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:03:04.852924  9313 tablet_bootstrap.cc:492] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:04.862389  9314 tablet_bootstrap.cc:492] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:04.862829  9313 tablet_bootstrap.cc:654] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.862969  9314 ts_tablet_manager.cc:1403] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.009s
I20260709 13:03:04.863878  9312 tablet_bootstrap.cc:492] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.864331  9312 ts_tablet_manager.cc:1403] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.028s	user 0.021s	sys 0.002s
I20260709 13:03:04.866123  9314 raft_consensus.cc:359] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.866762  9312 raft_consensus.cc:359] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.867131  9314 raft_consensus.cc:385] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.867455  9312 raft_consensus.cc:385] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.867810  9312 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.867666  9314 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.868707  9312 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.869081  9353 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.869153  9314 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.869720  9353 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.871104  9312 ts_tablet_manager.cc:1434] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:04.872794  9314 ts_tablet_manager.cc:1434] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:03:04.873891  9314 tablet_bootstrap.cc:492] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:04.873788  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:04.874688  9313 tablet_bootstrap.cc:492] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:04.875216  9313 ts_tablet_manager.cc:1403] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.006s
I20260709 13:03:04.874575  9271 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:04.877200  9087 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:04.877884  9312 tablet_bootstrap.cc:492] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:04.878163  9319 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.878577  9319 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.878502  9313 raft_consensus.cc:359] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.879122  9319 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.881372  9314 tablet_bootstrap.cc:654] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.884493  9312 tablet_bootstrap.cc:654] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.889197  9319 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.890544  9353 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.892848  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:04.893445  9198 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.893868  9314 tablet_bootstrap.cc:492] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:04.894382  9314 ts_tablet_manager.cc:1403] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.002s
I20260709 13:03:04.894260  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:04.897528  9314 raft_consensus.cc:359] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.898417  9314 raft_consensus.cc:385] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.899844  9314 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.900677  9314 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.904462  9314 ts_tablet_manager.cc:1434] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:03:04.901098  9198 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:04.905478  9314 tablet_bootstrap.cc:492] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:04.901546  9313 raft_consensus.cc:385] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.906023  9313 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.906745  9313 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.903003  9312 tablet_bootstrap.cc:492] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.907819  9312 ts_tablet_manager.cc:1403] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.004s
I20260709 13:03:04.908095  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:04.908371  9336 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.908823  9336 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.909276  9353 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:04.910475  9353 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:04.910965  9336 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:04.912068  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:04.910959  9319 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.913072  9198 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:04.912978  9353 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.913693  9313 ts_tablet_manager.cc:1434] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.004s
I20260709 13:03:04.912024  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:04.914541  9271 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.914541  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:04.916013  9336 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.916385  9336 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.916718  9336 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.910677  9312 raft_consensus.cc:359] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.918085  9312 raft_consensus.cc:385] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.918366  9312 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.919041  9312 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.920171  9319 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.920820  9319 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.922086  9271 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:04.912024  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:04.923728  9122 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:04.924130  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:04.923765  9312 ts_tablet_manager.cc:1434] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
W20260709 13:03:04.925784  9089 leader_election.cc:343] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.926199  9312 tablet_bootstrap.cc:492] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:04.927682  9319 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.912068  9314 tablet_bootstrap.cc:654] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.931262  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:04.932776  9087 leader_election.cc:343] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:04.933209  9087 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:04.934311  9319 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:04.934646  9329 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.935160  9329 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.935660  9342 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:04.936676  9312 tablet_bootstrap.cc:654] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.938354  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:04.938560  9329 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:04.939038  9122 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:04.941090  9353 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:04.940680  9020 catalog_manager.cc:5697] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:04.941985  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:04.942642  9272 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:04.941200  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:04.947258  9329 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:04.947716  9329 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.948149  9329 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.951406  9336 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.953867  9353 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:04.914608  9313 tablet_bootstrap.cc:492] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:04.964253  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:04.964488  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:04.964988  9122 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.964988  9196 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:04.971534  9312 tablet_bootstrap.cc:492] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:04.972249  9312 ts_tablet_manager.cc:1403] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.006s
I20260709 13:03:04.972131  9336 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:04.975097  9122 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:04.975153  9196 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:04.961131  9313 tablet_bootstrap.cc:654] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:04.975994  9312 raft_consensus.cc:359] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.976919  9312 raft_consensus.cc:385] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:04.976828  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:04.977422  9312 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:04.978228  9336 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:04.978569  9312 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.979339  9336 raft_consensus.cc:697] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:04.980883  9312 ts_tablet_manager.cc:1434] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:03:04.981878  9336 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:04.988492  9312 tablet_bootstrap.cc:492] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:05.005949  9312 tablet_bootstrap.cc:654] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.006804  9339 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:04.987380  9329 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.025849  9319 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.027072  9329 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.032927  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:05.033555  9272 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.042728  9272 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.044088  9162 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:05.045941  9313 tablet_bootstrap.cc:492] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.046444  9313 ts_tablet_manager.cc:1403] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.132s	user 0.009s	sys 0.009s
I20260709 13:03:05.049255  9313 raft_consensus.cc:359] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.049968  9313 raft_consensus.cc:385] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.050240  9313 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
W20260709 13:03:05.046463  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:05.052358  9329 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:05.052861  9329 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:05.053761  9329 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.055227  9020 catalog_manager.cc:5697] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:05.058525  9314 tablet_bootstrap.cc:492] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.059039  9314 ts_tablet_manager.cc:1403] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.154s	user 0.030s	sys 0.006s
I20260709 13:03:05.060472  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:05.061101  9122 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.064009  9314 raft_consensus.cc:359] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.064949  9314 raft_consensus.cc:385] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.065294  9314 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.066385  9314 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.067576  9313 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.069487  9314 ts_tablet_manager.cc:1434] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:03:05.070998  9313 ts_tablet_manager.cc:1434] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:03:05.074329  9122 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.074599  9339 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.082626  9313 tablet_bootstrap.cc:492] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.082752  9339 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.085752  9123 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.089124  9313 tablet_bootstrap.cc:654] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.089511  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.096719  9339 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.077099  9391 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.081146  9312 tablet_bootstrap.cc:492] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:05.099064  9312 ts_tablet_manager.cc:1403] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.111s	user 0.009s	sys 0.005s
I20260709 13:03:05.098307  9391 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.099576  9018 catalog_manager.cc:5697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.102581  9314 tablet_bootstrap.cc:492] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.106374  9312 raft_consensus.cc:359] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.107161  9312 raft_consensus.cc:385] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.108321  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.108508  9391 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.109932  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.107456  9312 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
W20260709 13:03:05.111650  9162 leader_election.cc:343] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.111997  9312 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:05.112941  9161 leader_election.cc:343] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.114161  9312 ts_tablet_manager.cc:1434] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
I20260709 13:03:05.114305  9314 tablet_bootstrap.cc:654] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.115043  9312 tablet_bootstrap.cc:492] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:05.114935  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.117323  9161 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:05.118009  9313 tablet_bootstrap.cc:492] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.118144  9391 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:05.118486  9313 ts_tablet_manager.cc:1403] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.008s
I20260709 13:03:05.118934  9089 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:05.119828  9339 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:05.120992  9313 raft_consensus.cc:359] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.121745  9313 raft_consensus.cc:385] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.122026  9313 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.122848  9313 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.125545  9313 ts_tablet_manager.cc:1434] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:03:05.125834  9391 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:05.126475  9313 tablet_bootstrap.cc:492] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.134385  9312 tablet_bootstrap.cc:654] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.134649  9336 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:05.134650  9313 tablet_bootstrap.cc:654] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.139217  9314 tablet_bootstrap.cc:492] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.139864  9314 ts_tablet_manager.cc:1403] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.005s
I20260709 13:03:05.143765  9314 raft_consensus.cc:359] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.144718  9314 raft_consensus.cc:385] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.145059  9314 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.146126  9314 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.152275  9314 ts_tablet_manager.cc:1434] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:03:05.161134  9338 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:05.166183  9314 tablet_bootstrap.cc:492] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.168538  9312 tablet_bootstrap.cc:492] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:05.171875  9312 ts_tablet_manager.cc:1403] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.007s
I20260709 13:03:05.174889  9314 tablet_bootstrap.cc:654] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.177757  9312 raft_consensus.cc:359] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.180574  9314 tablet_bootstrap.cc:492] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.182717  9312 raft_consensus.cc:385] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.181154  9313 tablet_bootstrap.cc:492] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.183099  9314 ts_tablet_manager.cc:1403] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:03:05.183526  9313 ts_tablet_manager.cc:1403] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.057s	user 0.010s	sys 0.018s
I20260709 13:03:05.186141  9314 raft_consensus.cc:359] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.186836  9314 raft_consensus.cc:385] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.186722  9313 raft_consensus.cc:359] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.187306  9314 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.187556  9313 raft_consensus.cc:385] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.187999  9313 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.183089  9312 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.188123  9314 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.189623  9313 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.189630  9312 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.192236  9314 ts_tablet_manager.cc:1434] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:03:05.193600  9314 tablet_bootstrap.cc:492] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.194849  9329 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.195353  9329 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.197539  9313 ts_tablet_manager.cc:1434] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:05.197671  9329 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.198426  9312 ts_tablet_manager.cc:1434] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:03:05.200155  9313 tablet_bootstrap.cc:492] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.200525  9314 tablet_bootstrap.cc:654] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.201274  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:05.202721  9162 leader_election.cc:343] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.209108  9313 tablet_bootstrap.cc:654] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.211993  9312 tablet_bootstrap.cc:492] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:05.216470  9329 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.217947  9329 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.219177  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.219918  9123 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:05.223204  9161 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:05.226053  9391 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:05.226490  9312 tablet_bootstrap.cc:654] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.226645  9391 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.227243  9391 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.230084  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.231063  9272 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:05.235407  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.238354  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:05.240455  9392 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:05.241106  9392 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:05.241134  9161 leader_election.cc:343] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.241730  9392 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.228135  9329 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.247900  9353 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:05.253185  9391 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.253423  9392 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.255251  9391 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.256070  9392 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.256035  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:05.256901  9123 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.257634  9314 tablet_bootstrap.cc:492] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.258014  9312 tablet_bootstrap.cc:492] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:05.258065  9314 ts_tablet_manager.cc:1403] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.065s	user 0.018s	sys 0.031s
I20260709 13:03:05.258503  9312 ts_tablet_manager.cc:1403] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.008s
I20260709 13:03:05.260869  9314 raft_consensus.cc:359] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.261008  9312 raft_consensus.cc:359] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.261572  9314 raft_consensus.cc:385] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.261605  9312 raft_consensus.cc:385] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.262004  9314 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.262053  9312 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.262032  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:05.262961  9270 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.263196  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:05.262974  9312 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:05.265290  9161 leader_election.cc:343] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.266067  9312 ts_tablet_manager.cc:1434] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:05.266268  9314 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.267653  9312 tablet_bootstrap.cc:492] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:05.268364  9313 tablet_bootstrap.cc:492] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.268868  9313 ts_tablet_manager.cc:1403] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.069s	user 0.030s	sys 0.009s
I20260709 13:03:05.271898  9313 raft_consensus.cc:359] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.272619  9313 raft_consensus.cc:385] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.272948  9313 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.274113  9312 tablet_bootstrap.cc:654] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.273656  9313 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.276221  9314 ts_tablet_manager.cc:1434] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:03:05.276944  9313 ts_tablet_manager.cc:1434] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:03:05.277248  9314 tablet_bootstrap.cc:492] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.278034  9313 tablet_bootstrap.cc:492] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.286298  9314 tablet_bootstrap.cc:654] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.292832  9313 tablet_bootstrap.cc:654] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.292677  9319 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.319478  9353 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.265995  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:03:05.323026  9162 leader_election.cc:343] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.324930  9313 tablet_bootstrap.cc:492] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.325517  9313 ts_tablet_manager.cc:1403] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.048s	user 0.028s	sys 0.009s
I20260709 13:03:05.328413  9314 tablet_bootstrap.cc:492] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.329103  9314 ts_tablet_manager.cc:1403] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.052s	user 0.019s	sys 0.020s
I20260709 13:03:05.332175  9313 raft_consensus.cc:359] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.333026  9313 raft_consensus.cc:385] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.333240  9314 raft_consensus.cc:359] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.334082  9313 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.334259  9314 raft_consensus.cc:385] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.334800  9314 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.335428  9313 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.336143  9314 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.337939  9313 ts_tablet_manager.cc:1434] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:03:05.339402  9314 ts_tablet_manager.cc:1434] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:03:05.342657  9313 tablet_bootstrap.cc:492] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.344012  9314 tablet_bootstrap.cc:492] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.350204  9313 tablet_bootstrap.cc:654] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.352877  9314 tablet_bootstrap.cc:654] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.360694  9319 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.365440  9314 tablet_bootstrap.cc:492] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.365564  9313 tablet_bootstrap.cc:492] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.366384  9313 ts_tablet_manager.cc:1403] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.005s
I20260709 13:03:05.366304  9123 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.367857  9161 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:05.366318  9314 ts_tablet_manager.cc:1403] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.006s
I20260709 13:03:05.369603  9392 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:05.369681  9313 raft_consensus.cc:359] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.370242  9392 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:05.371232  9392 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.372411  9314 raft_consensus.cc:359] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.373725  9314 raft_consensus.cc:385] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.373992  9314 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.374992  9338 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.375332  9270 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.375479  9338 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.376127  9313 raft_consensus.cc:385] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.376893  9313 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.377146  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:05.378226  9338 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:05.378553  9391 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:05.379259  9391 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:05.380400  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.380564  9391 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.381346  9122 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:05.380827  9314 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.378242  9313 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.384002  9314 ts_tablet_manager.cc:1434] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 13:03:05.385389  9314 tablet_bootstrap.cc:492] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.385758  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.386787  9196 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:05.388145  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:05.388628  9313 ts_tablet_manager.cc:1434] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:03:05.389866  9338 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.391407  9313 tablet_bootstrap.cc:492] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.399688  9329 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.400635  9338 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:05.400403  9329 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.404305  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.405184  9122 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:05.406777  9161 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:05.407920  9388 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:05.408416  9388 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.408887  9388 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.409704  9313 tablet_bootstrap.cc:654] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.421804  9388 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.423879  9329 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.430943  9314 tablet_bootstrap.cc:654] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.430642  9336 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.434520  9388 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.435670  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:03:05.437458  9162 leader_election.cc:343] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.439345  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:05.440107  9123 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.442183  9353 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.442688  9353 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.444985  9353 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.450944  9329 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.453557  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:05.455308  9162 leader_election.cc:343] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.457983  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:05.459842  9087 leader_election.cc:343] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.461529  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.463855  9329 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.467689  9329 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.468227  9329 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.470662  9336 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.471554  9329 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.478730  9123 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.480752  9353 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.481354  9353 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.492240  9312 tablet_bootstrap.cc:492] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:05.492270  9353 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.492852  9312 ts_tablet_manager.cc:1403] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.225s	user 0.100s	sys 0.112s
I20260709 13:03:05.493512  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.494367  9271 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:05.494788  9313 tablet_bootstrap.cc:492] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.495322  9313 ts_tablet_manager.cc:1403] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.104s	user 0.016s	sys 0.018s
I20260709 13:03:05.495532  9312 raft_consensus.cc:359] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.496438  9312 raft_consensus.cc:385] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.496685  9312 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.497356  9312 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.498204  9313 raft_consensus.cc:359] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.498965  9313 raft_consensus.cc:385] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.499135  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.499248  9313 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.499670  9312 ts_tablet_manager.cc:1434] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:03:05.500275  9313 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.500742  9312 tablet_bootstrap.cc:492] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
W20260709 13:03:05.500810  9089 leader_election.cc:343] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.501359  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:05.502084  9353 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:05.502454  9313 ts_tablet_manager.cc:1434] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:03:05.503470  9313 tablet_bootstrap.cc:492] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.505203  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.505846  9122 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:05.509924  9312 tablet_bootstrap.cc:654] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.511709  9313 tablet_bootstrap.cc:654] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.514794  9272 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:05.516322  9162 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:05.517675  9392 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:05.518117  9392 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.518522  9392 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.518294  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.522744  9161 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:05.523501  9329 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:05.524101  9329 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:05.524662  9312 tablet_bootstrap.cc:492] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:05.524595  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:05.525182  9312 ts_tablet_manager.cc:1403] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.007s
I20260709 13:03:05.525074  9329 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.525815  9353 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.526376  9313 tablet_bootstrap.cc:492] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.526826  9313 ts_tablet_manager.cc:1403] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.002s
I20260709 13:03:05.534186  9313 raft_consensus.cc:359] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.535029  9313 raft_consensus.cc:385] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.535367  9313 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.536352  9313 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.538319  9312 raft_consensus.cc:359] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.539076  9312 raft_consensus.cc:385] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.539392  9312 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.540128  9312 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.531407  9392 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.540624  9353 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:05.543936  9312 ts_tablet_manager.cc:1434] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:03:05.544095  9319 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.544909  9312 tablet_bootstrap.cc:492] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:05.544740  9319 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.546480  9392 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.548175  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:05.548899  9270 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.551703  9312 tablet_bootstrap.cc:654] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.552812  9319 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.553448  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:05.554986  9087 leader_election.cc:343] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.554997  9313 ts_tablet_manager.cc:1434] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:03:05.556747  9313 tablet_bootstrap.cc:492] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.559084  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:05.559851  9123 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.566870  9313 tablet_bootstrap.cc:654] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.573807  9392 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.574693  9392 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.579186  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.579991  9271 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:05.580366  9314 tablet_bootstrap.cc:492] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.581038  9314 ts_tablet_manager.cc:1403] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.196s	user 0.013s	sys 0.005s
I20260709 13:03:05.581453  9313 tablet_bootstrap.cc:492] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.581636  9162 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:05.581945  9313 ts_tablet_manager.cc:1403] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.005s
I20260709 13:03:05.582540  9391 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.583031  9391 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.585314  9391 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.585162  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.586128  9391 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:05.586546  9391 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.586944  9391 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.587491  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:05.589044  9162 leader_election.cc:343] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.586983  9392 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.573103  9123 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.595263  9391 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.595286  9314 raft_consensus.cc:359] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.585280  9313 raft_consensus.cc:359] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.597100  9314 raft_consensus.cc:385] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.597766  9314 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.598706  9314 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.597345  9313 raft_consensus.cc:385] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.599938  9313 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.600221  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:05.601006  9269 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.600831  9313 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.603248  9313 ts_tablet_manager.cc:1434] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 13:03:05.604375  9313 tablet_bootstrap.cc:492] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.606235  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.608563  9314 ts_tablet_manager.cc:1434] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.002s
I20260709 13:03:05.608822  9391 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.609177  9423 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.610455  9314 tablet_bootstrap.cc:492] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.612929  9123 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:05.609930  9423 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.612178  9313 tablet_bootstrap.cc:654] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.617709  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.618687  9122 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:05.620520  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:05.621034  9314 tablet_bootstrap.cc:654] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.621882  9338 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:05.612792  9269 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.622416  9338 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.622946  9338 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.623782  9162 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:05.624881  9391 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:05.625512  9391 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:05.626714  9391 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.633149  9423 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:05.634815  9314 tablet_bootstrap.cc:492] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.635468  9314 ts_tablet_manager.cc:1403] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.008s
I20260709 13:03:05.635542  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:05.637791  9089 leader_election.cc:343] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.638485  9089 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:05.639933  9319 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:05.639921  9314 raft_consensus.cc:359] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.641139  9314 raft_consensus.cc:385] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.641491  9314 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.642450  9314 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.642952  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:05.643813  9122 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.645004  9314 ts_tablet_manager.cc:1434] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:03:05.646157  9314 tablet_bootstrap.cc:492] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.645089  9313 tablet_bootstrap.cc:492] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.646730  9313 ts_tablet_manager.cc:1403] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.001s
I20260709 13:03:05.650247  9313 raft_consensus.cc:359] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.651063  9313 raft_consensus.cc:385] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.651407  9313 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.652464  9314 tablet_bootstrap.cc:654] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.652175  9313 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.654575  9313 ts_tablet_manager.cc:1434] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:05.655814  9313 tablet_bootstrap.cc:492] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.656700  9018 catalog_manager.cc:5697] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.658896  9423 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.659616  9423 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.659430  9018 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.662427  9423 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:05.663878  9314 tablet_bootstrap.cc:492] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.664470  9314 ts_tablet_manager.cc:1403] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 13:03:05.664294  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.665403  9121 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:05.667161  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:05.667523  9314 raft_consensus.cc:359] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.668288  9313 tablet_bootstrap.cc:654] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.668336  9314 raft_consensus.cc:385] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.668334  9423 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:05.668962  9314 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.669123  9423 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.669678  9423 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.669963  9314 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.674265  9270 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.676127  9162 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:05.677335  9391 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:05.677973  9391 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:05.679095  9391 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.687762  9423 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.699049  9423 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:05.691816  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.699038  9338 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.704057  9196 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
W20260709 13:03:05.694151  9161 leader_election.cc:343] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.696175  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.690279  9314 ts_tablet_manager.cc:1434] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.006s
I20260709 13:03:05.700317  9336 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.707465  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:05.708029  9336 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.708339  9121 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.708308  9161 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:05.709362  9391 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:05.710251  9312 tablet_bootstrap.cc:492] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:05.710711  9312 ts_tablet_manager.cc:1403] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.166s	user 0.065s	sys 0.085s
I20260709 13:03:05.710942  9336 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:05.712211  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.712692  9314 tablet_bootstrap.cc:492] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.713472  9120 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:05.701386  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:05.717234  9123 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.717952  9312 raft_consensus.cc:359] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.718675  9312 raft_consensus.cc:385] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.718969  9312 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.707686  9313 tablet_bootstrap.cc:492] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.720193  9313 ts_tablet_manager.cc:1403] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.004s
W20260709 13:03:05.723381  9238 leader_election.cc:343] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.724618  9121 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:05.724853  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:05.725564  9123 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:05.726173  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:05.726747  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.727236  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:05.723668  9313 raft_consensus.cc:359] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.727924  9313 raft_consensus.cc:385] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.728243  9313 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
W20260709 13:03:05.728513  9238 leader_election.cc:343] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.729540  9122 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.730124  9238 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:05.719652  9312 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.730937  9313 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.731848  9314 tablet_bootstrap.cc:654] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.732337  9312 ts_tablet_manager.cc:1434] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:03:05.733405  9313 ts_tablet_manager.cc:1434] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:03:05.735679  9313 tablet_bootstrap.cc:492] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.734004  9439 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:05.736289  9439 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:05.733400  9312 tablet_bootstrap.cc:492] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:05.743080  9313 tablet_bootstrap.cc:654] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.742883  9440 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.734938  9338 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:05.733399  9423 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:05.749917  9391 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.751542  9423 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:05.751996  9391 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.752574  9423 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.783324  9312 tablet_bootstrap.cc:654] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.780050  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:05.781184  9391 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.737126  9439 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.792443  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.789153  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:05.800726  9161 leader_election.cc:343] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:05.802170  9162 leader_election.cc:343] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.802628  9162 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:05.795001  9198 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.804822  9313 tablet_bootstrap.cc:492] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.805339  9313 ts_tablet_manager.cc:1403] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.070s	user 0.021s	sys 0.013s
I20260709 13:03:05.807014  9391 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:05.795321  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 68168.4us
I20260709 13:03:05.820170  9313 raft_consensus.cc:359] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.821381  9313 raft_consensus.cc:385] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.822098  9313 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.823081  9313 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.825970  9313 ts_tablet_manager.cc:1434] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:03:05.827175  9313 tablet_bootstrap.cc:492] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.829337  9392 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.829277  9018 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.831610  9018 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.839087  9313 tablet_bootstrap.cc:654] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.849740  9440 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:05.850337  9198 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:05.853009  9339 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.853348  9392 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:03:05.855003  9314 tablet_bootstrap.cc:492] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.855572  9314 ts_tablet_manager.cc:1403] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.143s	user 0.018s	sys 0.007s
I20260709 13:03:05.854449  9339 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.858835  9314 raft_consensus.cc:359] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.859954  9314 raft_consensus.cc:385] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.860280  9314 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.860742  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.861527  9339 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.862280  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.862967  9314 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:05.865566  9089 leader_election.cc:343] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.866459  9269 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:05.868304  9087 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:05.869474  9314 ts_tablet_manager.cc:1434] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:03:05.870312  9339 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:05.870684  9314 tablet_bootstrap.cc:492] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:05.870743  9339 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.871317  9339 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.873425  9312 tablet_bootstrap.cc:492] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:05.874032  9312 ts_tablet_manager.cc:1403] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.141s	user 0.028s	sys 0.016s
I20260709 13:03:05.853880  9018 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:05.884662  9392 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:05.887851  9312 raft_consensus.cc:359] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.888562  9312 raft_consensus.cc:385] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.888835  9312 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.889768  9314 tablet_bootstrap.cc:654] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.891925  9329 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.892452  9329 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.896420  9329 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.899645  9353 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.899672  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.900485  9272 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:05.900323  9353 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.901904  9162 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:05.902637  9353 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.903957  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.904042  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.904345  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.904911  9122 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
W20260709 13:03:05.905659  9087 leader_election.cc:343] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.902976  9020 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:05.906193  9089 leader_election.cc:343] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.906441  9329 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:05.906607  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:05.906886  9329 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.907290  9329 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:05.898177  9312 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.907816  9020 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:05.911031  9312 ts_tablet_manager.cc:1434] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.002s
I20260709 13:03:05.912997  9312 tablet_bootstrap.cc:492] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:05.911181  9353 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:05.919289  9313 tablet_bootstrap.cc:492] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:05.919880  9313 ts_tablet_manager.cc:1403] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.005s
I20260709 13:03:05.920786  9392 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:05.922775  9313 raft_consensus.cc:359] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.923506  9313 raft_consensus.cc:385] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.923872  9313 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.924718  9313 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.927238  9329 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.927448  9313 ts_tablet_manager.cc:1434] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:03:05.930692  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:05.930896  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:05.890910  9339 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.931488  9122 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:05.931638  9270 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:05.930900  9313 tablet_bootstrap.cc:492] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:05.934078  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:03:05.936753  9089 leader_election.cc:343] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:05.930739  9440 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.937752  9440 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.941562  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:05.965056  9314 tablet_bootstrap.cc:492] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:05.977133  9314 ts_tablet_manager.cc:1403] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.107s	user 0.013s	sys 0.007s
I20260709 13:03:05.960225  9312 tablet_bootstrap.cc:654] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.944365  9329 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.983654  9313 tablet_bootstrap.cc:654] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:05.950984  9270 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:05.947026  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:05.968109  9339 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.969312  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:05.946344  9440 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:05.954377  9388 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 13:03:05.988615  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:05.948015  9392 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.989223  9314 raft_consensus.cc:359] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.991169  9314 raft_consensus.cc:385] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:05.991391  9314 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:05.989418  9269 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:05.990729  9392 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:05.996454  9392 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:05.997889  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:05.998610  9272 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:05.999794  9314 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.002851  9314 ts_tablet_manager.cc:1434] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:03:05.999163  9440 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:06.003521  9162 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:06.004133  9314 tablet_bootstrap.cc:492] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:06.010008  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.010800  9121 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:06.012748  9392 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:06.013206  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:06.013355  9392 raft_consensus.cc:697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:06.014504  9392 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.015941  9391 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:06.016395  9391 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.016798  9391 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.023278  9122 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:06.049397  9314 tablet_bootstrap.cc:654] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.051009  9269 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:06.052130  9087 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:06.054198  9454 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:06.054738  9454 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:06.053624  9391 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.055687  9454 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:06.061146  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:06.064304  9391 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.068773  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:06.069432  9123 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.070891  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:06.071728  9312 tablet_bootstrap.cc:492] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:06.072376  9312 ts_tablet_manager.cc:1403] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.160s	user 0.011s	sys 0.007s
I20260709 13:03:06.073173  9021 catalog_manager.cc:5697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 2, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:06.075013  9269 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.076766  9312 raft_consensus.cc:359] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.078733  9319 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.079677  9270 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.079499  9319 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.079807  9312 raft_consensus.cc:385] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.080549  9312 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.081913  9312 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.082700  9319 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.083215  9123 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:06.085476  9312 ts_tablet_manager.cc:1434] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:03:06.086472  9312 tablet_bootstrap.cc:492] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:06.090057  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:06.093918  9087 leader_election.cc:343] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.084954  9391 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:03:06.104921  9423 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.105425  9423 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.106436  9391 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:06.110162  9313 tablet_bootstrap.cc:492] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:06.110683  9313 ts_tablet_manager.cc:1403] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.180s	user 0.014s	sys 0.028s
I20260709 13:03:06.110610  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.108850  9423 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:06.111479  9123 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:06.112766  9235 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:06.113833  9423 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:06.114215  9423 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.114616  9423 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.116283  9313 raft_consensus.cc:359] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.117216  9313 raft_consensus.cc:385] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.117570  9313 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.118393  9313 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.133630  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:06.136386  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:06.137104  9198 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:06.143294  9463 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:06.144937  9462 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:03:06.160506  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:06.161303  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:06.146014  9463 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:03:06.189002  9238 leader_election.cc:343] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.189679  9270 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:06.181794  9339 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.192952  9339 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.194550  9339 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.185702  9319 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:06.195655  9319 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.196197  9319 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.200389  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:06.201678  9087 leader_election.cc:343] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.207829  9319 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.209604  9319 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.208434  9019 catalog_manager.cc:5697] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:06.210870  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:06.212148  9312 tablet_bootstrap.cc:654] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:06.212143  9087 leader_election.cc:343] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.213095  9462 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:06.214761  9465 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.084s	user 0.002s	sys 0.000s
I20260709 13:03:06.166298  9463 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.218683  9313 ts_tablet_manager.cc:1434] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.108s	user 0.001s	sys 0.031s
I20260709 13:03:06.221202  9313 tablet_bootstrap.cc:492] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
W20260709 13:03:06.216415  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 53964.6us
I20260709 13:03:06.230358  9391 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:06.233817  9314 tablet_bootstrap.cc:492] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:06.234366  9314 ts_tablet_manager.cc:1403] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.231s	user 0.012s	sys 0.008s
I20260709 13:03:06.241734  9314 raft_consensus.cc:359] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.242825  9314 raft_consensus.cc:385] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.243952  9314 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.239995  9339 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.245203  9339 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.244935  9314 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.247362  9339 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.247500  9314 ts_tablet_manager.cc:1434] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:03:06.248854  9314 tablet_bootstrap.cc:492] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:06.250980  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:06.254696  9087 leader_election.cc:343] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.249217  9423 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.264634  9313 tablet_bootstrap.cc:654] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.267733  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:06.268148  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:06.268921  9122 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.268127  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:06.269019  9197 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.270890  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:06.271828  9423 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:03:06.289013  9089 leader_election.cc:343] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.289634  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:06.290738  9339 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:06.290167  9019 catalog_manager.cc:5697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:06.290413  9442 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:06.293823  9089 leader_election.cc:343] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.294374  9089 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:06.290510  9423 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:06.296351  9339 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:06.301712  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:06.317317  9122 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
W20260709 13:03:06.318413  9238 leader_election.cc:343] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.309693  9197 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:06.319641  9336 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:06.326432  9312 tablet_bootstrap.cc:492] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:06.327160  9312 ts_tablet_manager.cc:1403] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.241s	user 0.013s	sys 0.010s
I20260709 13:03:06.332880  9319 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.333164  9235 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:06.333500  9319 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.334784  9336 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:06.335347  9336 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:06.335969  9319 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.341843  9312 raft_consensus.cc:359] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.342991  9312 raft_consensus.cc:385] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.343379  9312 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.346750  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:06.348049  9238 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:06.349347  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:06.350589  9319 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:06.348554  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:06.351765  9319 raft_consensus.cc:697] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:03:06.348892  9087 leader_election.cc:343] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.351368  9336 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.352716  9319 consensus_queue.cc:237] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.356588  9312 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.360371  9312 ts_tablet_manager.cc:1434] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.033s	user 0.002s	sys 0.006s
I20260709 13:03:06.361714  9312 tablet_bootstrap.cc:492] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:06.373409  9313 tablet_bootstrap.cc:492] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:06.374118  9313 ts_tablet_manager.cc:1403] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.154s	user 0.012s	sys 0.024s
I20260709 13:03:06.378084  9314 tablet_bootstrap.cc:654] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:06.394289  9089 leader_election.cc:343] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.394850  9089 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:06.394584  9021 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.395574  9454 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:06.398924  9312 tablet_bootstrap.cc:654] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.402276  9313 raft_consensus.cc:359] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.402961  9313 raft_consensus.cc:385] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.403226  9313 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.403949  9313 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.406030  9313 ts_tablet_manager.cc:1434] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 13:03:06.410503  9314 tablet_bootstrap.cc:492] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:06.411311  9314 ts_tablet_manager.cc:1403] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.163s	user 0.014s	sys 0.000s
I20260709 13:03:06.413347  9312 tablet_bootstrap.cc:492] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:06.413878  9312 ts_tablet_manager.cc:1403] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.003s
I20260709 13:03:06.414415  9314 raft_consensus.cc:359] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.415123  9314 raft_consensus.cc:385] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.415380  9314 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.416055  9314 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.416466  9312 raft_consensus.cc:359] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.417100  9312 raft_consensus.cc:385] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.417341  9312 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.406929  9313 tablet_bootstrap.cc:492] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:06.418175  9312 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.420881  9312 ts_tablet_manager.cc:1434] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:03:06.421875  9312 tablet_bootstrap.cc:492] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:06.418202  9314 ts_tablet_manager.cc:1434] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:03:06.425784  9314 tablet_bootstrap.cc:492] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:06.432015  9312 tablet_bootstrap.cc:654] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.434664  9018 catalog_manager.cc:5697] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:06.437668  9313 tablet_bootstrap.cc:654] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.439043  9314 tablet_bootstrap.cc:654] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.445561  9312 tablet_bootstrap.cc:492] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:06.445991  9312 ts_tablet_manager.cc:1403] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.010s
I20260709 13:03:06.448469  9336 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.449014  9336 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.451157  9336 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:06.451190  9312 raft_consensus.cc:359] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.452116  9312 raft_consensus.cc:385] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.452421  9312 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.452920  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.453150  9312 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.455304  9312 ts_tablet_manager.cc:1434] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:03:06.455353  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:06.456108  9195 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:06.456435  9312 tablet_bootstrap.cc:492] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:06.453606  9123 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:06.458220  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:06.459064  9462 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.459698  9477 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:06.459522  9462 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.460119  9477 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.460477  9477 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.462402  9462 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.463766  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:06.463973  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.464617  9270 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:06.465808  9162 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:06.466504  9462 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.468843  9314 tablet_bootstrap.cc:492] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:06.468523  9477 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.469336  9314 ts_tablet_manager.cc:1403] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.044s	user 0.004s	sys 0.006s
I20260709 13:03:06.470851  9477 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:06.472025  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:06.472558  9123 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.473994  9314 raft_consensus.cc:359] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.474663  9314 raft_consensus.cc:385] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.474852  9312 tablet_bootstrap.cc:654] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.474946  9314 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.475721  9314 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.476156  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:06.476760  9195 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.477684  9314 ts_tablet_manager.cc:1434] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:03:06.478623  9314 tablet_bootstrap.cc:492] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:06.486538  9314 tablet_bootstrap.cc:654] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.488976  9312 tablet_bootstrap.cc:492] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:06.489596  9312 ts_tablet_manager.cc:1403] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.005s
I20260709 13:03:06.492484  9195 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:06.492967  9312 raft_consensus.cc:359] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.496340  9312 raft_consensus.cc:385] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.496052  9123 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:06.496785  9312 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.496762  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:06.497875  9336 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:06.497649  9312 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.498337  9336 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:06.499152  9336 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.499778  9312 ts_tablet_manager.cc:1434] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:03:06.500403  9462 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:06.501458  9312 tablet_bootstrap.cc:492] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:06.503784  9314 tablet_bootstrap.cc:492] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:06.504786  9314 ts_tablet_manager.cc:1403] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.008s
I20260709 13:03:06.505772  9313 tablet_bootstrap.cc:492] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:06.506227  9313 ts_tablet_manager.cc:1403] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.099s	user 0.010s	sys 0.001s
I20260709 13:03:06.507298  9312 tablet_bootstrap.cc:654] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.507449  9314 raft_consensus.cc:359] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.508153  9314 raft_consensus.cc:385] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.508448  9314 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.508944  9313 raft_consensus.cc:359] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.509112  9314 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.509732  9313 raft_consensus.cc:385] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.510067  9313 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.510919  9313 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.513243  9313 ts_tablet_manager.cc:1434] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:03:06.516516  9313 tablet_bootstrap.cc:492] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:06.518467  9312 tablet_bootstrap.cc:492] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:06.518936  9312 ts_tablet_manager.cc:1403] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 13:03:06.521541  9312 raft_consensus.cc:359] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.522217  9312 raft_consensus.cc:385] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.522480  9312 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.523300  9312 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.524129  9314 ts_tablet_manager.cc:1434] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:03:06.525017  9314 tablet_bootstrap.cc:492] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:06.525575  9313 tablet_bootstrap.cc:654] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.526356  9270 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:06.528167  9461 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:06.528416  9454 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.529047  9454 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.530936  9314 tablet_bootstrap.cc:654] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.534857  9312 ts_tablet_manager.cc:1434] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:03:06.536139  9312 tablet_bootstrap.cc:492] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:06.536746  9336 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.537487  9336 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.541567  9336 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:06.543292  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.544143  9123 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:06.544420  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:06.545042  9195 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:06.545524  9235 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:06.546450  9336 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:06.546953  9336 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.547456  9336 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.547677  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:06.548439  9195 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:06.550834  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:06.553105  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:06.554510  9339 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:06.555099  9339 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.555598  9339 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.559652  9391 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.560391  9391 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.561280  9123 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:06.564293  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.564693  9312 tablet_bootstrap.cc:654] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:06.565927  9161 leader_election.cc:343] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.565742  9392 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:06.568459  9314 tablet_bootstrap.cc:492] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:06.568944  9314 ts_tablet_manager.cc:1403] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.000s
I20260709 13:03:06.569718  9391 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:03:06.571465  9087 leader_election.cc:343] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.546468  9454 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.571952  9314 raft_consensus.cc:359] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.572926  9314 raft_consensus.cc:385] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.573292  9314 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.574110  9314 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.579826  9336 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.579627  9019 catalog_manager.cc:5697] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.582911  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:06.583608  9123 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.584698  9336 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:06.590699  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:06.591435  9195 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.580168  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:06.597584  9314 ts_tablet_manager.cc:1434] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
W20260709 13:03:06.597929  9162 leader_election.cc:343] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.599992  9162 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:06.601738  9391 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:06.605546  9314 tablet_bootstrap.cc:492] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:06.606367  9123 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:06.607556  9235 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:06.608582  9336 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:06.609175  9336 raft_consensus.cc:697] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:06.610499  9195 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:06.610209  9336 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.612361  9339 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.615568  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:06.616302  9198 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.618707  9314 tablet_bootstrap.cc:654] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.634967  9198 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:06.638412  9339 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.639389  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:06.639410  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:03:06.641086  9087 leader_election.cc:343] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.642287  9313 tablet_bootstrap.cc:492] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:06.641187  9339 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:06.642859  9313 ts_tablet_manager.cc:1403] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.127s	user 0.023s	sys 0.037s
I20260709 13:03:06.643088  9339 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:06.643389  9454 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:06.643381  9019 catalog_manager.cc:5697] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.644307  9339 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.646168  9313 raft_consensus.cc:359] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.647192  9313 raft_consensus.cc:385] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.647500  9313 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.647964  9312 tablet_bootstrap.cc:492] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:06.648411  9312 ts_tablet_manager.cc:1403] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.112s	user 0.019s	sys 0.018s
I20260709 13:03:06.648274  9313 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.650403  9313 ts_tablet_manager.cc:1434] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:03:06.650964  9312 raft_consensus.cc:359] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.651567  9312 raft_consensus.cc:385] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.653282  9312 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.654592  9314 tablet_bootstrap.cc:492] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:06.654983  9314 ts_tablet_manager.cc:1403] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.000s
I20260709 13:03:06.657586  9313 tablet_bootstrap.cc:492] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:06.664650  9313 tablet_bootstrap.cc:654] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.666977  9312 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.657429  9314 raft_consensus.cc:359] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.668645  9314 raft_consensus.cc:385] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.669005  9314 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.671762  9312 ts_tablet_manager.cc:1434] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:03:06.673838  9314 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.676419  9018 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:06.679636  9314 ts_tablet_manager.cc:1434] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:03:06.680182  9313 tablet_bootstrap.cc:492] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:06.680599  9314 tablet_bootstrap.cc:492] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:06.680677  9313 ts_tablet_manager.cc:1403] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 13:03:06.673020  9312 tablet_bootstrap.cc:492] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:06.690407  9312 tablet_bootstrap.cc:654] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.694597  9314 tablet_bootstrap.cc:654] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.694432  9319 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:06.696072  9462 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.696681  9462 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.700488  9462 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.704092  9313 raft_consensus.cc:359] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.704947  9313 raft_consensus.cc:385] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.705317  9313 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.706115  9313 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.709336  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:06.714445  9313 ts_tablet_manager.cc:1434] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.005s
I20260709 13:03:06.714509  9312 tablet_bootstrap.cc:492] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:06.715302  9312 ts_tablet_manager.cc:1403] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.003s
I20260709 13:03:06.715814  9313 tablet_bootstrap.cc:492] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:06.718185  9312 raft_consensus.cc:359] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.718856  9312 raft_consensus.cc:385] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.719153  9312 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.720052  9312 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.722646  9313 tablet_bootstrap.cc:654] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:06.725088  9162 leader_election.cc:343] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.728350  9312 ts_tablet_manager.cc:1434] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.005s
I20260709 13:03:06.729633  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.730381  9120 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:06.730602  9312 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:06.732887  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:06.733917  9391 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:06.734344  9391 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.734717  9391 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.737259  9462 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.737905  9462 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.742036  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:06.743897  9162 leader_election.cc:343] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.753465  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.754508  9120 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:06.757316  9462 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.759702  9462 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.761808  9161 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:06.760425  9462 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.766387  9195 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.766171  9391 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.768199  9392 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:06.768790  9392 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.769325  9392 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.778209  9392 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.781581  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:06.782284  9120 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.799353  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.799670  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:06.802657  9161 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:06.803493  9462 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.804337  9477 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.807487  9477 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.811820  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.813091  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:03:06.814906  9162 leader_election.cc:343] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:06.815284  9235 leader_election.cc:343] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.815971  9477 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:06.817503  9339 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:06.818437  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:06.819031  9123 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.819219  9120 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:06.827066  9123 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:06.828044  9391 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.828191  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:03:06.829989  9162 leader_election.cc:343] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.832736  9312 tablet_bootstrap.cc:654] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:06.835050  9463 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:06.828997  9392 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.836616  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:03:06.840552  9238 leader_election.cc:343] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:06.841162  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:06.841497  9339 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.842036  9339 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.842361  9477 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:06.844177  9339 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.846674  9161 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:06.847249  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:06.848204  9391 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:06.848120  9269 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:06.848806  9463 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:06.849831  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:06.851163  9087 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:06.851986  9339 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:06.848825  9391 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:06.852533  9339 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.852959  9339 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.853376  9391 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.851996  9463 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:06.860635  9463 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:06.861974  9463 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.885674  9314 tablet_bootstrap.cc:492] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:06.886664  9314 ts_tablet_manager.cc:1403] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.206s	user 0.072s	sys 0.083s
I20260709 13:03:06.892328  9339 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.892807  9477 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:06.897609  9313 tablet_bootstrap.cc:492] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:06.898362  9313 ts_tablet_manager.cc:1403] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.183s	user 0.038s	sys 0.040s
I20260709 13:03:06.900008  9197 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.899946  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:06.901118  9198 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:06.912176  9313 raft_consensus.cc:359] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.914575  9313 raft_consensus.cc:385] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:06.915006  9313 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:06.916042  9313 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.922822  9313 ts_tablet_manager.cc:1434] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:03:06.924826  9313 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:06.909503  9391 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:03:06.937496  9339 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.941540  9019 catalog_manager.cc:5697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:06.946009  9019 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:06.949385  9423 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:03:06.953713  9463 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.954236  9463 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.956509  9463 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:06.957279  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:06.957864  9197 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.961484  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:06.962355  9270 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:06.977958  9477 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.978694  9477 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:06.981571  9477 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:06.976262  9336 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.987370  9336 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:07.050101  9506 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:03:07.016999  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:06.973483  9423 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.065s
I20260709 13:03:07.084069  9314 raft_consensus.cc:359] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.085151  9314 raft_consensus.cc:385] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.085512  9314 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.088037  9314 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.092617  9314 ts_tablet_manager.cc:1434] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.205s	user 0.008s	sys 0.000s
I20260709 13:03:07.102856  9336 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:07.108105  9312 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:07.107892  9503 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.108635  9312 ts_tablet_manager.cc:1403] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.378s	user 0.045s	sys 0.071s
I20260709 13:03:07.108661  9503 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.113570  9503 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.113860  9312 raft_consensus.cc:359] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.114552  9312 raft_consensus.cc:385] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.114789  9312 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.115286  9312 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.120388  9462 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.120975  9462 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.123020  9462 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:03:07.102774  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 109569us
I20260709 13:03:07.124992  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.129897  9197 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:07.130281  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.131585  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.131675  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.134205  9313 tablet_bootstrap.cc:654] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.139132  9270 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:07.142171  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.144013  9161 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:07.145741  9312 ts_tablet_manager.cc:1434] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:03:07.149197  9120 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:07.152626  9312 tablet_bootstrap.cc:492] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:07.153746  9271 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:07.155036  9477 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:03:07.164255  9269 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.172798  9235 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:07.174028  9477 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:07.174081  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.174676  9477 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.175192  9477 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.146557  9319 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.176523  9319 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.184531  9162 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:07.186429  9391 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:07.187623  9391 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.187989  9391 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.196980  9462 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:07.192389  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48467.3us
I20260709 13:03:07.202293  9319 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.211395  9314 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
W20260709 13:03:07.219596  9087 leader_election.cc:343] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.216090  9515 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.223196  9515 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.228161  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.228991  9120 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:07.232725  9391 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.240424  9515 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:07.242427  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.243177  9120 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:07.244253  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.250412  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.251788  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:07.251933  9087 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:07.254762  9516 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:07.256687  9477 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.257346  9516 raft_consensus.cc:697] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:07.258226  9516 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.260625  9312 tablet_bootstrap.cc:654] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.262213  9391 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.263245  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:07.264075  9272 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:07.258848  9477 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:07.265676  9515 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:07.268813  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:07.269433  9121 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.270471  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.274391  9313 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:07.275202  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.280052  9193 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:07.279932  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.280947  9313 ts_tablet_manager.cc:1403] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.357s	user 0.016s	sys 0.009s
I20260709 13:03:07.281986  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:07.282636  9391 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.283159  9391 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:07.284350  9087 leader_election.cc:343] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.286562  9312 tablet_bootstrap.cc:492] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:07.287181  9319 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:07.287671  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:07.278888  9514 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:07.286985  9089 leader_election.cc:343] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.287062  9312 ts_tablet_manager.cc:1403] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.135s	user 0.011s	sys 0.008s
I20260709 13:03:07.288339  9089 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:07.289081  9503 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:07.277552  9314 tablet_bootstrap.cc:654] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.290972  9312 raft_consensus.cc:359] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.291709  9312 raft_consensus.cc:385] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.287497  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.292028  9312 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
W20260709 13:03:07.293018  9162 leader_election.cc:343] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.292851  9312 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.293885  9503 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:07.295631  9312 ts_tablet_manager.cc:1434] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 13:03:07.292116  9391 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.296582  9312 tablet_bootstrap.cc:492] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
W20260709 13:03:07.295912  9089 leader_election.cc:343] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.303808  9313 raft_consensus.cc:359] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.304457  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:07.306061  9319 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:07.306250  9314 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:07.306725  9314 ts_tablet_manager.cc:1403] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.095s	user 0.010s	sys 0.000s
I20260709 13:03:07.309378  9314 raft_consensus.cc:359] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.304752  9313 raft_consensus.cc:385] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.310027  9314 raft_consensus.cc:385] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.301632  9161 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:07.310413  9313 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.310814  9391 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:07.314940  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:07.315712  9198 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:07.320858  9312 tablet_bootstrap.cc:654] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.319403  9313 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.324082  9313 ts_tablet_manager.cc:1434] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 13:03:07.331923  9313 tablet_bootstrap.cc:492] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:07.310971  9121 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:07.314206  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.335299  9119 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:07.335498  9477 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:07.337327  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:07.310413  9314 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.326023  9329 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:07.340984  9463 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:07.341534  9463 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.341879  9463 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.342545  9235 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:07.339378  9314 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.344465  9515 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:07.345103  9515 raft_consensus.cc:697] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:07.346153  9313 tablet_bootstrap.cc:654] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.346199  9515 consensus_queue.cc:237] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.350600  9463 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.353217  9314 ts_tablet_manager.cc:1434] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:03:07.354231  9314 tablet_bootstrap.cc:492] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:07.354338  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:07.358796  9477 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.360970  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:07.361608  9271 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.370410  9463 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.369424  9271 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:07.372145  9162 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:07.373268  9477 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:07.373543  9463 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:07.374231  9463 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:07.375135  9463 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:07.398371  9087 consensus_peers.cc:597] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:07.404526  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:07.415099  9122 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.440215  9313 tablet_bootstrap.cc:492] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:07.441138  9313 ts_tablet_manager.cc:1403] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.109s	user 0.012s	sys 0.006s
I20260709 13:03:07.441277  9391 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:07.448341  9122 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:07.464459  9314 tablet_bootstrap.cc:654] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.453768  9018 catalog_manager.cc:5697] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:07.456601  9312 tablet_bootstrap.cc:492] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:07.472261  9312 ts_tablet_manager.cc:1403] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.176s	user 0.035s	sys 0.044s
W20260709 13:03:07.457401  9162 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:07.477276  9312 raft_consensus.cc:359] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.477972  9312 raft_consensus.cc:385] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.478245  9312 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.478866  9312 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.480953  9312 ts_tablet_manager.cc:1434] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:03:07.481431  9314 tablet_bootstrap.cc:492] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:07.458441  9313 raft_consensus.cc:359] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.460762  9462 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:07.482160  9313 raft_consensus.cc:385] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.482191  9314 ts_tablet_manager.cc:1403] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.128s	user 0.017s	sys 0.003s
I20260709 13:03:07.482574  9313 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.483363  9313 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.485366  9312 tablet_bootstrap.cc:492] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:07.485447  9313 ts_tablet_manager.cc:1434] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.001s
I20260709 13:03:07.486336  9313 tablet_bootstrap.cc:492] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:07.490409  9021 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:07.497347  9314 raft_consensus.cc:359] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.498126  9314 raft_consensus.cc:385] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.498386  9314 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.499078  9314 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.501123  9314 ts_tablet_manager.cc:1434] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:03:07.501909  9314 tablet_bootstrap.cc:492] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:07.507505  9312 tablet_bootstrap.cc:654] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.514861  9313 tablet_bootstrap.cc:654] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.515285  9314 tablet_bootstrap.cc:654] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.511510  9019 catalog_manager.cc:5697] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:07.567782  9313 tablet_bootstrap.cc:492] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:07.568529  9313 ts_tablet_manager.cc:1403] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.082s	user 0.017s	sys 0.037s
I20260709 13:03:07.568609  9314 tablet_bootstrap.cc:492] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:07.569190  9314 ts_tablet_manager.cc:1403] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.029s
I20260709 13:03:07.571475  9313 raft_consensus.cc:359] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.572330  9313 raft_consensus.cc:385] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.572167  9314 raft_consensus.cc:359] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.572724  9313 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.573010  9314 raft_consensus.cc:385] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.573352  9314 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.573781  9313 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.574441  9314 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.575806  9313 ts_tablet_manager.cc:1434] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:07.576753  9313 tablet_bootstrap.cc:492] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:07.576954  9314 ts_tablet_manager.cc:1434] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:07.577925  9314 tablet_bootstrap.cc:492] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:07.582888  9313 tablet_bootstrap.cc:654] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.583813  9314 tablet_bootstrap.cc:654] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.584636  9312 tablet_bootstrap.cc:492] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:07.585201  9312 ts_tablet_manager.cc:1403] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.104s	user 0.016s	sys 0.022s
I20260709 13:03:07.588871  9312 raft_consensus.cc:359] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.589792  9312 raft_consensus.cc:385] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.590065  9312 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.590752  9312 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.592869  9313 tablet_bootstrap.cc:492] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:07.593122  9312 ts_tablet_manager.cc:1434] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:07.593497  9313 ts_tablet_manager.cc:1403] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:03:07.596279  9313 raft_consensus.cc:359] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.597039  9313 raft_consensus.cc:385] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.597370  9313 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.598124  9313 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.598752  9314 tablet_bootstrap.cc:492] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:07.599215  9314 ts_tablet_manager.cc:1403] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:03:07.600244  9313 ts_tablet_manager.cc:1434] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:07.600427  9312 tablet_bootstrap.cc:492] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:07.601249  9313 tablet_bootstrap.cc:492] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:07.601847  9314 raft_consensus.cc:359] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.602557  9314 raft_consensus.cc:385] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.602856  9314 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.603516  9314 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.606001  9314 ts_tablet_manager.cc:1434] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:03:07.607241  9314 tablet_bootstrap.cc:492] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:07.608281  9313 tablet_bootstrap.cc:654] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.610620  9336 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.611275  9336 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.612084  9312 tablet_bootstrap.cc:654] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.613443  9336 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:07.615134  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.617337  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.619303  9238 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:07.625948  9336 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:07.628860  9314 tablet_bootstrap.cc:654] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.645620  9314 tablet_bootstrap.cc:492] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:07.646129  9314 ts_tablet_manager.cc:1403] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.006s
I20260709 13:03:07.648873  9314 raft_consensus.cc:359] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.649605  9314 raft_consensus.cc:385] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.649899  9314 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.650594  9314 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.653508  9314 ts_tablet_manager.cc:1434] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:07.655346  9314 tablet_bootstrap.cc:492] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:07.661779  9314 tablet_bootstrap.cc:654] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.673602  9313 tablet_bootstrap.cc:492] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:07.674177  9313 ts_tablet_manager.cc:1403] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.073s	user 0.032s	sys 0.021s
I20260709 13:03:07.674606  9314 tablet_bootstrap.cc:492] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:07.675096  9314 ts_tablet_manager.cc:1403] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:03:07.676885  9313 raft_consensus.cc:359] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.677682  9313 raft_consensus.cc:385] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.677970  9313 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.677896  9314 raft_consensus.cc:359] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.678725  9314 raft_consensus.cc:385] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.679108  9314 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.679359  9312 tablet_bootstrap.cc:492] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:07.678783  9313 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.679836  9312 ts_tablet_manager.cc:1403] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.080s	user 0.022s	sys 0.024s
I20260709 13:03:07.679834  9314 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.686533  9314 ts_tablet_manager.cc:1434] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.004s
I20260709 13:03:07.688025  9314 tablet_bootstrap.cc:492] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:07.690099  9312 raft_consensus.cc:359] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.690851  9312 raft_consensus.cc:385] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.691144  9312 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.691833  9312 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.696179  9336 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.696779  9336 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.698269  9516 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:07.703958  9336 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:07.705794  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.706893  9312 ts_tablet_manager.cc:1434] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:03:07.706658  9122 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:07.707087  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.707836  9194 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:07.708195  9235 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:07.709196  9336 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:07.709664  9336 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.710109  9336 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.713732  9312 tablet_bootstrap.cc:492] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:07.718422  9313 ts_tablet_manager.cc:1434] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 13:03:07.719595  9313 tablet_bootstrap.cc:492] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:07.726217  9314 tablet_bootstrap.cc:654] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.732820  9312 tablet_bootstrap.cc:654] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.742475  9313 tablet_bootstrap.cc:654] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.750917  9462 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.751699  9462 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.756598  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.757336  9122 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:07.758611  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.760604  9313 tablet_bootstrap.cc:492] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:07.761243  9313 ts_tablet_manager.cc:1403] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.014s
I20260709 13:03:07.761873  9462 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.776958  9336 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.779537  9336 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:07.764815  9313 raft_consensus.cc:359] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.780230  9313 raft_consensus.cc:385] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.780520  9313 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.769773  9314 tablet_bootstrap.cc:492] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:07.781412  9314 ts_tablet_manager.cc:1403] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.094s	user 0.020s	sys 0.008s
I20260709 13:03:07.771437  9503 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.783089  9503 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.784173  9314 raft_consensus.cc:359] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.784912  9314 raft_consensus.cc:385] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.785192  9314 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.785270  9503 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.786026  9314 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.758636  9161 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:03:07.787855  9162 leader_election.cc:343] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.788169  9313 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.790668  9391 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:07.791090  9391 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.791450  9391 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.793902  9392 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.794683  9392 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.796130  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.796813  9194 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:07.799096  9391 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.800942  9392 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.803906  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:07.805267  9454 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:07.805171  9516 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:07.805444  9312 tablet_bootstrap.cc:492] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:07.805714  9454 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.806131  9454 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.806197  9312 ts_tablet_manager.cc:1403] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.097s	user 0.043s	sys 0.018s
I20260709 13:03:07.807714  9314 ts_tablet_manager.cc:1434] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:03:07.808636  9314 tablet_bootstrap.cc:492] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:07.809347  9312 raft_consensus.cc:359] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.802314  9313 ts_tablet_manager.cc:1434] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.002s
I20260709 13:03:07.802546  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.810132  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:07.809383  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.811481  9313 tablet_bootstrap.cc:492] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:07.812150  9336 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:07.814355  9162 leader_election.cc:343] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.810163  9312 raft_consensus.cc:385] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:03:07.815088  9162 leader_election.cc:343] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.815140  9312 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.816042  9312 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.822212  9312 ts_tablet_manager.cc:1434] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:03:07.823323  9312 tablet_bootstrap.cc:492] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:07.828987  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.820535  9313 tablet_bootstrap.cc:654] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.822002  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:07.829833  9121 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.830700  9269 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.829668  9272 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:07.831281  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:07.831949  9119 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:07.835466  9161 leader_election.cc:343] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.836088  9161 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:07.837806  9462 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:07.819236  9314 tablet_bootstrap.cc:654] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.841233  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:07.841936  9194 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.845453  9121 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:07.850544  9235 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:07.851137  9119 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:07.851879  9339 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.829263  9312 tablet_bootstrap.cc:654] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.853722  9515 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:07.828702  9454 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.852819  9161 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:07.854826  9463 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:07.855847  9463 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:07.857096  9463 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.852591  9339 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.861559  9454 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.871446  9314 tablet_bootstrap.cc:492] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:07.863552  9339 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.864826  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.871976  9314 ts_tablet_manager.cc:1403] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.064s	user 0.017s	sys 0.003s
I20260709 13:03:07.872126  9271 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:07.873546  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:07.865913  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:07.874516  9272 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.874912  9314 raft_consensus.cc:359] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.875623  9314 raft_consensus.cc:385] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.875913  9314 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.876713  9314 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.879334  9314 ts_tablet_manager.cc:1434] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:07.867228  9503 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.880067  9503 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.868032  9336 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:07.881521  9516 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:07.881918  9516 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.882256  9516 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.882422  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:07.883103  9197 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.880388  9314 tablet_bootstrap.cc:492] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:07.883524  9336 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.884066  9336 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.867789  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.887651  9336 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:07.854745  9515 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:07.888353  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.889087  9515 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.889595  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.892334  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:07.893105  9271 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:07.896199  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.896670  9087 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:03:07.897701  9235 leader_election.cc:343] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:07.897790  9089 leader_election.cc:343] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:07.898828  9089 leader_election.cc:343] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.899681  9194 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:07.900089  9454 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:07.900454  9454 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:07.900520  9238 leader_election.cc:343] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:07.900875  9454 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.801283  9391 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.902034  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:07.902763  9336 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:07.882251  9503 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:07.892663  9314 tablet_bootstrap.cc:654] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:07.913973  9272 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:07.916925  9516 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.920766  9087 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:07.922513  9197 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:07.925705  9312 tablet_bootstrap.cc:492] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:07.926213  9312 ts_tablet_manager.cc:1403] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.103s	user 0.009s	sys 0.003s
I20260709 13:03:07.928098  9339 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:07.940268  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:07.941107  9021 catalog_manager.cc:5697] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:07.946277  9423 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.947103  9423 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.948436  9312 raft_consensus.cc:359] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.950806  9312 raft_consensus.cc:385] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:07.951392  9312 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.955082  9312 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.957227  9312 ts_tablet_manager.cc:1434] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.002s
I20260709 13:03:07.958154  9312 tablet_bootstrap.cc:492] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:07.936549  9339 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:07.961082  9339 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.964705  9423 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:07.966089  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:07.966784  9121 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:07.968108  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:07.968981  9477 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:07.969375  9477 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.969729  9477 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:07.970922  9454 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.974452  9314 tablet_bootstrap.cc:492] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:07.975023  9314 ts_tablet_manager.cc:1403] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.095s	user 0.010s	sys 0.009s
I20260709 13:03:07.979215  9313 tablet_bootstrap.cc:492] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:07.979792  9313 ts_tablet_manager.cc:1403] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.169s	user 0.028s	sys 0.010s
I20260709 13:03:07.981952  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:07.982803  9195 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:07.983191  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:07.991199  9477 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.992444  9391 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:07.999022  9313 raft_consensus.cc:359] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:07.995774  9314 raft_consensus.cc:359] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.000082  9313 raft_consensus.cc:385] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.000375  9314 raft_consensus.cc:385] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.001133  9313 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.001303  9314 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:07.985508  9269 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.030066  9312 tablet_bootstrap.cc:654] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.041741  9314 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.030959  9313 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.002251  9516 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:08.035684  9454 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:08.056816  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:03:08.036921  9089 leader_election.cc:343] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:08.057626  9194 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.059002  9313 ts_tablet_manager.cc:1434] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.001s
I20260709 13:03:08.060354  9314 ts_tablet_manager.cc:1434] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.080s	user 0.009s	sys 0.000s
I20260709 13:03:08.064983  9392 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:08.066043  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:08.066675  9197 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.035279  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:08.071420  9314 tablet_bootstrap.cc:492] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:08.034723  9477 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:08.063050  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:08.077298  9268 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.071866  9272 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.078907  9122 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.106678  9269 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:08.114567  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:08.117790  9454 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:08.118381  9454 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:08.120036  9454 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.121172  9122 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:08.113425  9017 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:08.123478  9313 tablet_bootstrap.cc:492] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:08.123935  9197 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:08.125355  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:08.126993  9336 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:08.128069  9336 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:08.132805  9336 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.134665  9194 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:08.144402  9312 tablet_bootstrap.cc:492] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:08.145345  9312 ts_tablet_manager.cc:1403] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.187s	user 0.018s	sys 0.009s
I20260709 13:03:08.146966  9314 tablet_bootstrap.cc:654] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.151793  9313 tablet_bootstrap.cc:654] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.154527  9272 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:08.156418  9087 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:08.157986  9312 raft_consensus.cc:359] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.159021  9312 raft_consensus.cc:385] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.159355  9312 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.174254  9339 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.175290  9516 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:08.175428  9339 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.177969  9312 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.182735  9516 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:08.185771  9516 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.189960  9312 ts_tablet_manager.cc:1434] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.003s
I20260709 13:03:08.190819  9312 tablet_bootstrap.cc:492] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:08.193146  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:08.194198  9272 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:08.195164  9339 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:08.196933  9087 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:08.199846  9339 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:08.200397  9339 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.203258  9312 tablet_bootstrap.cc:654] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.204265  9339 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.203714  9313 tablet_bootstrap.cc:492] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:08.208081  9313 ts_tablet_manager.cc:1403] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.085s	user 0.008s	sys 0.011s
I20260709 13:03:08.207708  9021 catalog_manager.cc:5697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:08.210753  9313 raft_consensus.cc:359] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.211452  9313 raft_consensus.cc:385] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.211726  9313 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.217376  9313 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.219709  9313 ts_tablet_manager.cc:1434] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 13:03:08.220638  9313 tablet_bootstrap.cc:492] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:08.228996  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:08.229775  9197 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:08.229470  9339 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.233855  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:08.233909  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:08.234472  9197 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.234860  9272 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.236490  9339 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:08.241609  9313 tablet_bootstrap.cc:654] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.248598  9312 tablet_bootstrap.cc:492] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:08.249145  9312 ts_tablet_manager.cc:1403] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.013s
I20260709 13:03:08.252383  9312 raft_consensus.cc:359] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.253120  9312 raft_consensus.cc:385] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.253428  9312 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.254247  9312 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.256929  9312 ts_tablet_manager.cc:1434] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:08.257673  9272 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:08.257962  9312 tablet_bootstrap.cc:492] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:08.259114  9087 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:08.260309  9339 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:08.261055  9339 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:08.259714  9018 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:08.261909  9339 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.263037  9018 catalog_manager.cc:5697] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:08.264076  9314 tablet_bootstrap.cc:492] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:08.264544  9314 ts_tablet_manager.cc:1403] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.193s	user 0.014s	sys 0.036s
I20260709 13:03:08.267321  9314 raft_consensus.cc:359] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.281594  9314 raft_consensus.cc:385] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.281936  9314 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.282634  9314 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.284835  9314 ts_tablet_manager.cc:1434] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.003s
I20260709 13:03:08.285735  9314 tablet_bootstrap.cc:492] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:08.271317  9197 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:08.285938  9463 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:08.278662  9312 tablet_bootstrap.cc:654] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.279773  9313 tablet_bootstrap.cc:492] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:08.292179  9392 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.292735  9313 ts_tablet_manager.cc:1403] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.072s	user 0.008s	sys 0.010s
I20260709 13:03:08.292819  9392 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.294905  9392 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:08.295442  9314 tablet_bootstrap.cc:654] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.296054  9313 raft_consensus.cc:359] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.296736  9313 raft_consensus.cc:385] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.297008  9313 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.297629  9313 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.302887  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:08.304729  9313 ts_tablet_manager.cc:1434] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:03:08.306018  9313 tablet_bootstrap.cc:492] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
W20260709 13:03:08.306082  9162 leader_election.cc:343] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:08.308128  9314 tablet_bootstrap.cc:492] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:08.308764  9314 ts_tablet_manager.cc:1403] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.007s
I20260709 13:03:08.312134  9314 raft_consensus.cc:359] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.312788  9313 tablet_bootstrap.cc:654] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.312963  9314 raft_consensus.cc:385] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.313391  9314 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.314323  9314 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.317312  9314 ts_tablet_manager.cc:1434] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:03:08.318707  9314 tablet_bootstrap.cc:492] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:08.329452  9314 tablet_bootstrap.cc:654] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.341835  9314 tablet_bootstrap.cc:492] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:08.343513  9314 ts_tablet_manager.cc:1403] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.000s
I20260709 13:03:08.347110  9314 raft_consensus.cc:359] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.352037  9314 raft_consensus.cc:385] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.352430  9314 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.353543  9314 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.356572  9314 ts_tablet_manager.cc:1434] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:03:08.360455  9314 tablet_bootstrap.cc:492] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:08.364998  9314 tablet_bootstrap.cc:654] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.383198  9314 tablet_bootstrap.cc:492] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:08.383797  9314 ts_tablet_manager.cc:1403] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.005s
I20260709 13:03:08.388067  9314 raft_consensus.cc:359] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.389004  9314 raft_consensus.cc:385] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.389350  9314 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.390232  9314 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.393143  9314 ts_tablet_manager.cc:1434] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:03:08.394383  9314 tablet_bootstrap.cc:492] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:08.402135  9454 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.404435  9314 tablet_bootstrap.cc:654] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.404405  9454 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.403637  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:08.419620  9314 tablet_bootstrap.cc:492] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:08.420274  9314 ts_tablet_manager.cc:1403] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.008s
I20260709 13:03:08.426270  9314 raft_consensus.cc:359] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.427199  9314 raft_consensus.cc:385] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.427558  9314 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.428481  9314 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.431772  9314 ts_tablet_manager.cc:1434] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:03:08.433058  9314 tablet_bootstrap.cc:492] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:08.435148  9454 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:08.436292  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:08.438259  9087 leader_election.cc:343] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:08.413153  9121 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:08.441316  9314 tablet_bootstrap.cc:654] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:08.442857  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 93293.7us
I20260709 13:03:08.449585  9514 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.450126  9514 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.453747  9514 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:08.453907  9314 tablet_bootstrap.cc:492] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:08.454713  9314 ts_tablet_manager.cc:1403] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.005s
I20260709 13:03:08.455421  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:08.456348  9119 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:08.458155  9235 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:08.458722  9314 raft_consensus.cc:359] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.460045  9514 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:08.459653  9314 raft_consensus.cc:385] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.460703  9514 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.461772  9514 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.460867  9314 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.464986  9314 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.467989  9314 ts_tablet_manager.cc:1434] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 13:03:08.469310  9314 tablet_bootstrap.cc:492] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:08.476025  9272 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.474968  9021 catalog_manager.cc:5697] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:08.491004  9021 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:08.498270  9314 tablet_bootstrap.cc:654] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:08.501794  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:08.510229  9336 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.510939  9336 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.517505  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:08.522127  9336 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:08.542021  9462 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:08.546738  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:08.549777  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:08.550429  9193 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:08.552748  9515 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:08.556778  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:08.562495  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:03:08.564407  9238 leader_election.cc:343] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:08.565690  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:08.572187  9313 tablet_bootstrap.cc:492] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:08.572939  9313 ts_tablet_manager.cc:1403] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.267s	user 0.064s	sys 0.125s
I20260709 13:03:08.572214  9312 tablet_bootstrap.cc:492] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:08.574153  9312 ts_tablet_manager.cc:1403] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.316s	user 0.078s	sys 0.138s
I20260709 13:03:08.572959  9514 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.577900  9514 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:08.573761  9462 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:08.578908  9312 raft_consensus.cc:359] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.582870  9312 raft_consensus.cc:385] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.583246  9312 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.582062  9462 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.586196  9462 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.575317  9339 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:08.696615  9339 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.697383  9339 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.586522  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:03:08.679337  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 116775us
I20260709 13:03:08.689493  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:08.703243  9120 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:08.609831  9477 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:08.720533  9319 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:03:08.587107  9312 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.708478  9313 raft_consensus.cc:359] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.729411  9391 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.732923  9391 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.605696  9514 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.124s
I20260709 13:03:08.730154  9313 raft_consensus.cc:385] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:08.742897  9477 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.740067  9313 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:08.792718  9313 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:08.780681  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 76002.3us
I20260709 13:03:08.824093  9314 tablet_bootstrap.cc:492] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:08.825253  9314 ts_tablet_manager.cc:1403] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.356s	user 0.032s	sys 0.042s
I20260709 13:03:08.786494  9477 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.831142  9312 ts_tablet_manager.cc:1434] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.257s	user 0.006s	sys 0.027s
I20260709 13:03:08.832238  9454 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.833259  9312 tablet_bootstrap.cc:492] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:08.832964  9454 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.798034  9319 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.876108  9238 leader_election.cc:343] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:08.877140  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 54379.4us
I20260709 13:03:08.903462  9336 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.906860  9477 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:03:08.884730  9089 consensus_peers.cc:597] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:08.890302  9319 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.916213  9391 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:08.916868  9319 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:03:08.930115  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:08.930912  9120 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:08.935927  9462 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.939308  9462 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:08.941318  9313 ts_tablet_manager.cc:1434] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.368s	user 0.012s	sys 0.008s
I20260709 13:03:08.945509  9515 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.946309  9515 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.954424  9588 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.982554  9313 tablet_bootstrap.cc:492] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:08.955263  9588 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:08.987166  9336 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.041088  9339 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.087879  9339 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:09.090308  9336 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:09.092603  9588 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:09.093884  9454 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:09.094214  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:09.100579  9269 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:09.104764  9336 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:09.113080  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:09.119349  9313 tablet_bootstrap.cc:654] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:09.122504  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:09.124989  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:09.126521  9193 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:09.127066  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:09.131901  9314 raft_consensus.cc:359] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.133001  9314 raft_consensus.cc:385] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:09.135948  9314 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:09.139518  9515 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:09.140268  9312 tablet_bootstrap.cc:654] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:09.157164  9162 leader_election.cc:343] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.163105  9314 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.175307  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:09.174464  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:09.200892  9515 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.201499  9515 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.204704  9515 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:09.204675  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:09.205803  9197 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:03:09.217067  9238 leader_election.cc:343] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.220472  9314 ts_tablet_manager.cc:1434] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.395s	user 0.008s	sys 0.016s
I20260709 13:03:09.211305  9195 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
W20260709 13:03:09.239789  9238 leader_election.cc:343] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.236315  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:09.228425  9196 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:09.242141  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:09.230206  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:09.245966  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:09.247898  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:09.248411  9515 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:09.248697  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:09.252154  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:09.252806  9118 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:09.250281  9515 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.254475  9515 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:09.256007  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:09.256822  9116 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:09.248904  9122 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:09.277787  9235 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:09.279295  9336 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:09.279865  9336 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:09.281201  9336 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.295047  9313 tablet_bootstrap.cc:492] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:09.295591  9313 ts_tablet_manager.cc:1403] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.352s	user 0.018s	sys 0.003s
I20260709 13:03:09.297693  9515 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.301198  9588 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:09.321720  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:03:09.302865  9235 leader_election.cc:343] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.322568  9235 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:09.323107  9515 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:09.323824  9515 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:09.324523  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:09.324909  9235 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:09.325232  9271 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
W20260709 13:03:09.326280  9235 leader_election.cc:343] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:09.327255  9235 leader_election.cc:343] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.303198  9313 raft_consensus.cc:359] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.327939  9588 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:09.328017  9313 raft_consensus.cc:385] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:09.328460  9588 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.328553  9313 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:09.328933  9588 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:09.329627  9313 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.304315  9314 tablet_bootstrap.cc:492] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:09.332115  9313 ts_tablet_manager.cc:1434] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
I20260709 13:03:09.305918  9312 tablet_bootstrap.cc:492] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:09.332774  9312 ts_tablet_manager.cc:1403] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.500s	user 0.024s	sys 0.052s
I20260709 13:03:09.333418  9313 tablet_bootstrap.cc:492] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:09.306679  9597 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:09.306771  9089 leader_election.cc:343] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.311692  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:09.319295  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:09.303733  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:09.336869  9089 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:09.337466  9272 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 1: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:09.340183  9454 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:09.340646  9454 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.341027  9454 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:09.350957  9596 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260709 13:03:09.365736  9087 leader_election.cc:343] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.366243  9087 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:09.367060  9339 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:09.367486  9087 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:09.374768  9339 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:09.380889  9161 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:09.380898  9591 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:09.385819  9591 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.386338  9591 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:09.389477  9319 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:09.388832  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:09.396185  9312 raft_consensus.cc:359] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.396813  9312 raft_consensus.cc:385] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:09.397086  9312 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:09.400321  9595 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:03:09.377658  9597 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:03:09.404378  9312 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.400446  9585 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:09.419462  9392 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:09.427033  9588 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.435462  9454 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.437152  9018 catalog_manager.cc:5697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:09.445492  9238 leader_election.cc:343] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.442358  9314 tablet_bootstrap.cc:654] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:09.456689  9238 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:03:09.458724  9238 leader_election.cc:343] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.459298  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:09.462121  9312 ts_tablet_manager.cc:1434] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.129s	user 0.006s	sys 0.002s
I20260709 13:03:09.465862  9336 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:09.476298  9514 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:09.487022  9312 tablet_bootstrap.cc:492] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:09.477941  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:09.493718  9454 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:09.486145  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:09.510089  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:09.511072  9271 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 1: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:09.511873  9122 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:09.512424  9313 tablet_bootstrap.cc:654] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:09.507943  9195 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:09.549185  9588 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:09.546483  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:09.573809  9089 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:09.579370  9595 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:09.581377  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:09.570443  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:03:09.607321  9609 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:03:09.620491  9597 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.621160  9597 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.622221  9195 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:09.673597  9514 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.678957  9514 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.674016  9597 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:09.704900  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:09.721107  9312 tablet_bootstrap.cc:654] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:09.699599  9591 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.830837  9123 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:09.840226  9313 tablet_bootstrap.cc:492] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:09.840772  9313 ts_tablet_manager.cc:1403] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.508s	user 0.045s	sys 0.103s
I20260709 13:03:09.850065  9314 tablet_bootstrap.cc:492] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:09.850566  9314 ts_tablet_manager.cc:1403] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.563s	user 0.040s	sys 0.070s
I20260709 13:03:09.861521  9313 raft_consensus.cc:359] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.862771  9313 raft_consensus.cc:385] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:09.863057  9313 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:09.864025  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:09.865725  9591 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:09.869218  9313 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.871272  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:09.872530  9266 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 1: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:09.873236  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:09.879949  9585 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.880570  9585 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.892671  9585 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:09.896028  9314 raft_consensus.cc:359] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.898912  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:09.900204  9271 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:09.901345  9314 raft_consensus.cc:385] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:09.902689  9314 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:09.903590  9314 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.907272  9514 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:09.908244  9312 tablet_bootstrap.cc:492] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:09.908463  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:09.908947  9312 ts_tablet_manager.cc:1403] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.430s	user 0.014s	sys 0.010s
I20260709 13:03:09.909607  9313 ts_tablet_manager.cc:1434] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.068s	user 0.004s	sys 0.006s
W20260709 13:03:09.912755  9087 leader_election.cc:343] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.902302  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:09.917299  9589 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:09.917793  9589 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.918162  9589 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:09.922678  9314 ts_tablet_manager.cc:1434] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.001s
I20260709 13:03:09.928668  9312 raft_consensus.cc:359] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.961856  9312 raft_consensus.cc:385] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:09.962144  9312 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:09.929158  9313 tablet_bootstrap.cc:492] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
W20260709 13:03:09.948433  9238 leader_election.cc:343] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.964164  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:09.924378  9314 tablet_bootstrap.cc:492] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
W20260709 13:03:09.937335  9089 leader_election.cc:343] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.973917  9089 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:09.974761  9319 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:09.974236  9238 consensus_peers.cc:597] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:09.977073  9588 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:09.977823  9588 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:09.983222  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:09.979148  9588 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:09.987068  9238 leader_election.cc:343] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.987952  9589 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.988917  9312 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:09.995407  9235 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:09.998605  9612 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:03:09.997555  9089 leader_election.cc:343] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:09.999812  9089 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:10.001139  9454 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:10.011821  9312 ts_tablet_manager.cc:1434] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.102s	user 0.009s	sys 0.001s
I20260709 13:03:10.012198  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.012327  9612 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:10.012917  9116 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:10.015501  9312 tablet_bootstrap.cc:492] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
W20260709 13:03:10.018273  9161 leader_election.cc:343] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.029986  9235 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:10.033809  9589 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:10.034910  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:10.035514  9271 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.037940  9313 tablet_bootstrap.cc:654] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.040105  9612 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:10.049536  9612 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.050035  9612 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.045442  9271 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:10.064097  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:10.065171  9589 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:10.065763  9589 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:10.071089  9312 tablet_bootstrap.cc:654] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.066708  9589 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.082177  9314 tablet_bootstrap.cc:654] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:10.075973  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 46124.2us
I20260709 13:03:10.108162  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:03:10.119258  9161 leader_election.cc:343] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.165395  9313 tablet_bootstrap.cc:492] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:10.166005  9313 ts_tablet_manager.cc:1403] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.237s	user 0.038s	sys 0.038s
I20260709 13:03:10.171372  9017 catalog_manager.cc:5697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:10.176731  9612 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.186120  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:10.186780  9122 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.186901  9313 raft_consensus.cc:359] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.187520  9588 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.187789  9313 raft_consensus.cc:385] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.188263  9313 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.188102  9588 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.185174  9612 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:10.190128  9018 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.193668  9314 tablet_bootstrap.cc:492] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:10.194182  9314 ts_tablet_manager.cc:1403] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.270s	user 0.017s	sys 0.011s
I20260709 13:03:10.194491  9313 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.198908  9313 ts_tablet_manager.cc:1434] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:03:10.202345  9313 tablet_bootstrap.cc:492] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:10.202850  9588 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:10.197057  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:10.205000  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.210049  9120 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:10.211629  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:10.207630  9122 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
W20260709 13:03:10.208397  9238 leader_election.cc:343] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.213060  9312 tablet_bootstrap.cc:492] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:10.213634  9312 ts_tablet_manager.cc:1403] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.198s	user 0.013s	sys 0.016s
W20260709 13:03:10.213680  9238 leader_election.cc:343] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.214852  9235 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:10.216950  9312 raft_consensus.cc:359] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.217742  9312 raft_consensus.cc:385] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.218068  9312 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.218688  9514 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:10.219169  9514 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.218786  9312 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.219543  9514 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.221201  9312 ts_tablet_manager.cc:1434] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:03:10.222786  9312 tablet_bootstrap.cc:492] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:10.217187  9235 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:10.221220  9314 raft_consensus.cc:359] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.226891  9314 raft_consensus.cc:385] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.227380  9314 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.227640  9477 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:10.235061  9477 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:10.236238  9477 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.229691  9313 tablet_bootstrap.cc:654] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.243252  9314 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.251082  9514 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.273393  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:10.274080  9118 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.274930  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:03:10.276789  9238 leader_election.cc:343] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.281600  9314 ts_tablet_manager.cc:1434] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.087s	user 0.004s	sys 0.004s
I20260709 13:03:10.281441  9018 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.282840  9314 tablet_bootstrap.cc:492] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:10.288599  9118 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:10.292840  9235 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:10.293798  9477 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:10.294339  9477 raft_consensus.cc:697] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:10.295221  9477 consensus_queue.cc:237] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.298113  9514 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:10.301488  9312 tablet_bootstrap.cc:654] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.309216  9313 tablet_bootstrap.cc:492] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:10.309767  9313 ts_tablet_manager.cc:1403] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.108s	user 0.029s	sys 0.027s
I20260709 13:03:10.312183  9314 tablet_bootstrap.cc:654] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:10.314330  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:10.315232  9313 raft_consensus.cc:359] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.316105  9313 raft_consensus.cc:385] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.316398  9313 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.317126  9313 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.317817  9312 tablet_bootstrap.cc:492] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:10.318318  9312 ts_tablet_manager.cc:1403] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.096s	user 0.011s	sys 0.001s
I20260709 13:03:10.321193  9313 ts_tablet_manager.cc:1434] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:03:10.321012  9312 raft_consensus.cc:359] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.321821  9312 raft_consensus.cc:385] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.322142  9312 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.322291  9313 tablet_bootstrap.cc:492] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:10.323005  9312 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.325531  9312 ts_tablet_manager.cc:1434] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:10.326571  9312 tablet_bootstrap.cc:492] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:10.332805  9312 tablet_bootstrap.cc:654] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.336340  9018 catalog_manager.cc:5697] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.346181  9312 tablet_bootstrap.cc:492] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:10.346817  9312 ts_tablet_manager.cc:1403] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.000s
I20260709 13:03:10.349717  9314 tablet_bootstrap.cc:492] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:10.350231  9314 ts_tablet_manager.cc:1403] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.068s	user 0.007s	sys 0.005s
I20260709 13:03:10.350500  9312 raft_consensus.cc:359] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.351257  9312 raft_consensus.cc:385] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.351549  9312 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.352317  9312 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.353003  9314 raft_consensus.cc:359] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.353681  9314 raft_consensus.cc:385] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.353962  9314 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.354849  9312 ts_tablet_manager.cc:1434] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:10.354684  9314 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.355894  9312 tablet_bootstrap.cc:492] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:10.358378  9314 ts_tablet_manager.cc:1434] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:03:10.360114  9314 tablet_bootstrap.cc:492] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:10.363131  9589 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.363687  9589 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.367278  9589 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:10.368167  9314 tablet_bootstrap.cc:654] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.368033  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:10.368774  9270 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:10.369043  9589 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.369599  9589 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.374497  9162 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:10.375522  9585 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:10.375900  9585 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.376243  9585 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.377588  9313 tablet_bootstrap.cc:654] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.379391  9589 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:10.381709  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:10.387187  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.393993  9161 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:10.393203  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.395704  9118 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:10.396010  9618 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:10.397713  9585 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.405756  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:10.406469  9120 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.407308  9312 tablet_bootstrap.cc:654] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.419175  9314 tablet_bootstrap.cc:492] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:10.419692  9314 ts_tablet_manager.cc:1403] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.060s	user 0.033s	sys 0.005s
I20260709 13:03:10.419039  9120 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:10.422253  9585 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:10.422546  9161 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:10.424582  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:10.425244  9271 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.427088  9313 tablet_bootstrap.cc:492] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:10.427840  9313 ts_tablet_manager.cc:1403] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.106s	user 0.013s	sys 0.018s
I20260709 13:03:10.429823  9314 raft_consensus.cc:359] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.430563  9314 raft_consensus.cc:385] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.430821  9314 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.431599  9314 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.432402  9195 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.433310  9314 ts_tablet_manager.cc:1434] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:03:10.426692  9312 tablet_bootstrap.cc:492] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:10.434466  9312 ts_tablet_manager.cc:1403] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.079s	user 0.015s	sys 0.003s
I20260709 13:03:10.437650  9312 raft_consensus.cc:359] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.438329  9312 raft_consensus.cc:385] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.438592  9312 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.439249  9312 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.440918  9314 tablet_bootstrap.cc:492] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:10.446246  9589 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:10.446883  9589 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:10.447885  9589 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.450343  9313 raft_consensus.cc:359] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.453620  9312 ts_tablet_manager.cc:1434] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:03:10.453723  9313 raft_consensus.cc:385] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.454136  9313 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.454833  9312 tablet_bootstrap.cc:492] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:10.455335  9313 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.457646  9313 ts_tablet_manager.cc:1434] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:03:10.461970  9314 tablet_bootstrap.cc:654] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.465998  9271 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:10.474751  9612 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:10.475332  9514 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.476574  9514 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.479005  9514 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:10.481464  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:03:10.483145  9238 leader_election.cc:343] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.475955  9618 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.483723  9618 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.493482  9313 tablet_bootstrap.cc:492] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:10.498946  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.514808  9312 tablet_bootstrap.cc:654] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.517117  9618 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:10.500516  9514 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:03:10.523792  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:10.530143  9271 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:10.526607  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.533511  9162 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:10.534544  9589 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:10.534976  9589 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.535343  9589 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:10.537335  9161 leader_election.cc:343] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.545181  9585 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.545739  9585 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.548007  9585 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:10.548943  9021 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:10.556712  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:10.519805  9120 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:10.565371  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.570291  9118 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:10.571765  9477 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.573417  9161 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:10.574910  9585 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.590103  9313 tablet_bootstrap.cc:654] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.604375  9314 tablet_bootstrap.cc:492] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:10.604885  9314 ts_tablet_manager.cc:1403] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.019s
I20260709 13:03:10.606330  9589 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.607825  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:10.612187  9314 raft_consensus.cc:359] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.613199  9314 raft_consensus.cc:385] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.613442  9314 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.614120  9314 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.616297  9612 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:10.616676  9612 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.617039  9612 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.617401  9589 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:10.582824  9477 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.625240  9314 ts_tablet_manager.cc:1434] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 13:03:10.632884  9314 tablet_bootstrap.cc:492] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:10.641160  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:10.644186  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:03:10.648404  9161 leader_election.cc:343] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.641955  9270 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.657725  9312 tablet_bootstrap.cc:492] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:10.658274  9312 ts_tablet_manager.cc:1403] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.204s	user 0.028s	sys 0.033s
I20260709 13:03:10.658738  9589 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:10.659226  9477 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:10.660995  9312 raft_consensus.cc:359] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.661711  9312 raft_consensus.cc:385] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.661978  9312 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.662639  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.663515  9119 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:10.662596  9312 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.667482  9612 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.686573  9312 ts_tablet_manager.cc:1434] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.002s
I20260709 13:03:10.708714  9314 tablet_bootstrap.cc:654] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.732292  9312 tablet_bootstrap.cc:492] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:10.732290  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:10.734598  9192 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:10.739308  9313 tablet_bootstrap.cc:492] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:10.740276  9313 ts_tablet_manager.cc:1403] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.248s	user 0.015s	sys 0.017s
I20260709 13:03:10.667572  9270 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:10.706827  9585 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:10.747926  9235 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:10.749675  9514 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.761216  9122 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.760535  9593 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.765641  9593 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.767707  9593 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:10.768699  9612 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:10.774022  9514 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:10.775584  9313 raft_consensus.cc:359] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.786314  9313 raft_consensus.cc:385] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.786875  9313 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.788470  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:10.789510  9119 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.796113  9514 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:10.804045  9313 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.821931  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.823050  9122 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:10.828433  9313 ts_tablet_manager.cc:1434] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.088s	user 0.004s	sys 0.004s
I20260709 13:03:10.832414  9339 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.833088  9339 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.824013  9514 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.844900  9339 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:10.846208  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:10.831499  9593 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
W20260709 13:03:10.851543  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 89699.7us
I20260709 13:03:10.882613  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:10.903029  9312 tablet_bootstrap.cc:654] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:10.852908  9313 tablet_bootstrap.cc:492] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:10.899775  9162 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:10.869386  9119 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:10.920619  9198 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.867421  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:10.900308  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:10.848187  9514 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.881767  9588 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 13:03:10.930721  9089 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:10.914458  9197 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:10.917872  9463 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:03:10.916677  9585 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:10.935329  9454 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:10.941725  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:10.948977  9585 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:10.953014  9585 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.951337  9588 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:10.956542  9588 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:10.957734  9314 tablet_bootstrap.cc:492] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:10.957360  9588 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.958196  9314 ts_tablet_manager.cc:1403] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.326s	user 0.019s	sys 0.021s
I20260709 13:03:10.961311  9514 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:10.962903  9314 raft_consensus.cc:359] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.963913  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:10.963658  9314 raft_consensus.cc:385] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:10.966410  9314 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:10.967299  9314 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:10.968412  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:10.970674  9314 ts_tablet_manager.cc:1434] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
W20260709 13:03:10.971769  9235 leader_election.cc:343] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.971616  9594 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:10.972566  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:03:10.978539  9238 leader_election.cc:343] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:10.979856  9238 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:10.982949  9314 tablet_bootstrap.cc:492] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:10.986929  9514 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:10.989781  9656 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:10.997377  9594 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:11.008705  9197 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:11.020327  9656 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.054153  9021 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.067811  9312 tablet_bootstrap.cc:492] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:11.068342  9312 ts_tablet_manager.cc:1403] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.336s	user 0.014s	sys 0.018s
I20260709 13:03:11.051926  9656 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.078778  9656 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.080318  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:11.081118  9269 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:11.083616  9162 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:11.084903  9656 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:11.085466  9656 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.085647  9454 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.086269  9454 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.088732  9454 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.093986  9313 tablet_bootstrap.cc:654] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.094726  9312 raft_consensus.cc:359] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.095507  9312 raft_consensus.cc:385] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.095875  9312 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.096710  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:11.097081  9312 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.087535  9656 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:11.098026  9270 raft_consensus.cc:2393] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:03:11.100188  9665 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:03:11.106901  9021 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:11.125864  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 81224.8us
I20260709 13:03:11.136160  9594 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.141599  9314 tablet_bootstrap.cc:654] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.144909  9312 ts_tablet_manager.cc:1434] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.000s
I20260709 13:03:11.148006  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:11.148718  9116 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:11.149451  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:11.143013  9594 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.160825  9585 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:11.163503  9313 tablet_bootstrap.cc:492] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.164098  9313 ts_tablet_manager.cc:1403] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.315s	user 0.019s	sys 0.016s
I20260709 13:03:11.165902  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:11.169049  9594 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:11.144824  9656 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.170459  9454 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.172049  9656 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.173112  9312 tablet_bootstrap.cc:492] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:11.176177  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:11.177186  9270 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:11.181207  9454 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:11.180907  9313 raft_consensus.cc:359] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.181706  9313 raft_consensus.cc:385] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.182045  9313 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.181883  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:11.184567  9235 leader_election.cc:343] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.192248  9515 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.192796  9515 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.193552  9312 tablet_bootstrap.cc:654] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.183130  9313 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.196553  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:11.197978  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:11.196951  9515 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:11.198750  9313 ts_tablet_manager.cc:1434] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
W20260709 13:03:11.199445  9235 leader_election.cc:343] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.200927  9270 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:11.202401  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:11.203169  9118 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:11.206106  9313 tablet_bootstrap.cc:492] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:11.211189  9162 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:11.213385  9314 tablet_bootstrap.cc:492] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
W20260709 13:03:11.213937  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 44568.9us
I20260709 13:03:11.214440  9656 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:11.220875  9656 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:11.217756  9312 tablet_bootstrap.cc:492] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:11.222036  9656 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.223022  9312 ts_tablet_manager.cc:1403] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.003s
I20260709 13:03:11.213953  9314 ts_tablet_manager.cc:1403] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.232s	user 0.018s	sys 0.009s
I20260709 13:03:11.226480  9314 raft_consensus.cc:359] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.227167  9314 raft_consensus.cc:385] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.226747  9312 raft_consensus.cc:359] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.227430  9314 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.227541  9312 raft_consensus.cc:385] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.227916  9312 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.228636  9312 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:11.231621  9238 leader_election.cc:343] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.232031  9313 tablet_bootstrap.cc:654] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.232187  9238 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:11.233464  9515 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:11.231827  9312 ts_tablet_manager.cc:1434] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:03:11.235526  9314 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.237769  9314 ts_tablet_manager.cc:1434] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:03:11.239833  9314 tablet_bootstrap.cc:492] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:11.243636  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:11.244495  9192 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:11.245474  9314 tablet_bootstrap.cc:654] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.248710  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:11.249886  9021 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 2, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.249895  9515 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:11.254050  9515 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.254529  9515 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.255882  9312 tablet_bootstrap.cc:492] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:11.266963  9312 tablet_bootstrap.cc:654] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.229473  9118 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:11.272248  9313 tablet_bootstrap.cc:492] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.282231  9312 tablet_bootstrap.cc:492] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:11.282753  9312 ts_tablet_manager.cc:1403] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 13:03:11.283923  9313 ts_tablet_manager.cc:1403] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.017s
I20260709 13:03:11.286526  9313 raft_consensus.cc:359] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.287205  9313 raft_consensus.cc:385] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.287469  9313 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.289784  9515 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.290429  9312 raft_consensus.cc:359] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.291126  9312 raft_consensus.cc:385] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.291388  9312 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.291955  9515 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:11.292032  9312 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.294126  9312 ts_tablet_manager.cc:1434] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:03:11.294759  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:11.295112  9312 tablet_bootstrap.cc:492] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
W20260709 13:03:11.296047  9235 leader_election.cc:343] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.288118  9313 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.297919  9313 ts_tablet_manager.cc:1434] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:03:11.298190  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:11.298825  9192 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.299660  9314 tablet_bootstrap.cc:492] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:11.300108  9314 ts_tablet_manager.cc:1403] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.011s
I20260709 13:03:11.300292  9313 tablet_bootstrap.cc:492] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:11.302688  9314 raft_consensus.cc:359] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.303390  9314 raft_consensus.cc:385] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.303680  9314 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.304334  9314 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.306160  9192 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:11.306522  9314 ts_tablet_manager.cc:1434] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:11.307536  9314 tablet_bootstrap.cc:492] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:11.308348  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:11.309746  9594 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:11.310640  9594 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:11.311513  9594 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.313320  9314 tablet_bootstrap.cc:654] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.306854  9313 tablet_bootstrap.cc:654] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.338292  9312 tablet_bootstrap.cc:654] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.339610  9314 tablet_bootstrap.cc:492] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:11.340286  9314 ts_tablet_manager.cc:1403] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.005s
I20260709 13:03:11.352288  9670 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:11.348171  9589 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.358435  9589 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.362165  9589 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.364205  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:11.365932  9265 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:11.369251  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:11.370163  9120 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:11.364976  9313 tablet_bootstrap.cc:492] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.371912  9313 ts_tablet_manager.cc:1403] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.018s
I20260709 13:03:11.373901  9161 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:03:11.377254  9298 tablet.cc:2406] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:03:11.377383  9656 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:11.380717  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:11.391397  9017 catalog_manager.cc:5697] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.394833  9514 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.395413  9514 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.394744  9656 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:11.397567  9514 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:11.402904  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:03:11.405893  9238 leader_election.cc:343] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.406260  9314 raft_consensus.cc:359] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.406917  9314 raft_consensus.cc:385] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.407176  9314 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.407794  9314 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.409982  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:11.410488  9314 ts_tablet_manager.cc:1434] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 13:03:11.411585  9314 tablet_bootstrap.cc:492] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
W20260709 13:03:11.411650  9235 leader_election.cc:343] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.412189  9235 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:11.413280  9594 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:11.416889  9313 raft_consensus.cc:359] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.425035  9313 raft_consensus.cc:385] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.425401  9313 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.430635  9314 tablet_bootstrap.cc:654] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.430392  9594 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:11.430575  9313 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.441092  9313 ts_tablet_manager.cc:1434] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 13:03:11.442469  9313 tablet_bootstrap.cc:492] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:11.474834  9589 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:11.483497  9339 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.484016  9339 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.484628  9313 tablet_bootstrap.cc:654] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.486016  9339 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.488183  9312 tablet_bootstrap.cc:492] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:11.488785  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:11.489698  9270 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:11.491110  9087 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:11.492012  9339 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:11.492431  9339 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.492794  9339 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.498445  9312 ts_tablet_manager.cc:1403] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.204s	user 0.021s	sys 0.030s
I20260709 13:03:11.501134  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:11.507603  9339 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.510212  9312 raft_consensus.cc:359] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.514626  9312 raft_consensus.cc:385] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.514991  9312 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.515597  9312 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.518042  9312 ts_tablet_manager.cc:1434] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:03:11.517856  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:11.518553  9270 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.519160  9312 tablet_bootstrap.cc:492] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:11.526003  9312 tablet_bootstrap.cc:654] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.559931  9477 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:11.564846  9339 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.566974  9313 tablet_bootstrap.cc:492] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.567514  9313 ts_tablet_manager.cc:1403] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.125s	user 0.021s	sys 0.005s
I20260709 13:03:11.567525  9270 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:11.570122  9087 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:11.570487  9313 raft_consensus.cc:359] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.570962  9339 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:11.571199  9313 raft_consensus.cc:385] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.571511  9313 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.571735  9339 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:11.572153  9313 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.573154  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:11.574369  9313 ts_tablet_manager.cc:1434] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
W20260709 13:03:11.574401  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45090.9us
I20260709 13:03:11.575534  9313 tablet_bootstrap.cc:492] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
W20260709 13:03:11.575558  9089 leader_election.cc:343] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.585197  9313 tablet_bootstrap.cc:654] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.572594  9339 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.587771  9477 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.593027  9477 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.596338  9477 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:03:11.588182  9089 leader_election.cc:343] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.592271  9314 tablet_bootstrap.cc:492] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:11.596557  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:11.598389  9116 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:11.599800  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:11.601249  9477 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:11.602062  9477 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.603367  9477 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.605646  9312 tablet_bootstrap.cc:492] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:11.597256  9314 ts_tablet_manager.cc:1403] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.186s	user 0.055s	sys 0.057s
I20260709 13:03:11.606148  9312 ts_tablet_manager.cc:1403] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.087s	user 0.032s	sys 0.024s
I20260709 13:03:11.609854  9313 tablet_bootstrap.cc:492] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.608762  9314 raft_consensus.cc:359] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.612092  9313 ts_tablet_manager.cc:1403] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.004s
I20260709 13:03:11.612298  9314 raft_consensus.cc:385] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.612712  9314 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.613265  9021 catalog_manager.cc:5697] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.615219  9313 raft_consensus.cc:359] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.615908  9313 raft_consensus.cc:385] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.616176  9313 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.610814  9312 raft_consensus.cc:359] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.617280  9312 raft_consensus.cc:385] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.617534  9312 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.618260  9314 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.618446  9313 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.620631  9313 ts_tablet_manager.cc:1434] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:03:11.621534  9313 tablet_bootstrap.cc:492] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:11.621562  9314 ts_tablet_manager.cc:1434] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:03:11.622741  9314 tablet_bootstrap.cc:492] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:11.618260  9312 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.629312  9312 ts_tablet_manager.cc:1434] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 13:03:11.630359  9312 tablet_bootstrap.cc:492] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:11.630818  9314 tablet_bootstrap.cc:654] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.637517  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:11.635962  9477 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.638216  9195 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:11.639358  9477 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:11.640304  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:11.641124  9197 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.640229  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:11.643002  9118 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.651974  9454 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.652707  9118 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:11.652558  9454 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.653708  9197 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:11.653940  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:11.654836  9594 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:11.654901  9454 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.657126  9313 tablet_bootstrap.cc:654] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.660082  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:11.660760  9197 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:11.660961  9314 tablet_bootstrap.cc:492] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:11.661581  9314 ts_tablet_manager.cc:1403] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.002s
I20260709 13:03:11.662340  9312 tablet_bootstrap.cc:654] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.662671  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:11.663578  9454 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:11.663968  9454 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.664345  9454 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.655567  9594 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:11.670799  9594 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.673301  9454 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.675685  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:11.681890  9313 tablet_bootstrap.cc:492] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.682092  9270 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:11.682396  9313 ts_tablet_manager.cc:1403] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.006s
I20260709 13:03:11.682629  9454 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.677378  9314 raft_consensus.cc:359] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.683445  9314 raft_consensus.cc:385] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.684648  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:11.686204  9314 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.688271  9314 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.695834  9314 ts_tablet_manager.cc:1434] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
W20260709 13:03:11.699805  9693 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:03:11.687412  9270 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.686707  9313 raft_consensus.cc:359] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.687176  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:11.700675  9314 tablet_bootstrap.cc:492] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:11.700979  9313 raft_consensus.cc:385] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.701131  9195 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.707152  9313 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.708019  9313 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.709611  9313 ts_tablet_manager.cc:1434] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:03:11.715298  9195 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:11.710656  9017 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.718070  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:11.719151  9313 tablet_bootstrap.cc:492] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:11.719288  9454 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:11.719823  9454 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:11.720626  9454 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.728745  9594 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.729324  9594 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.730957  9314 tablet_bootstrap.cc:654] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.732704  9313 tablet_bootstrap.cc:654] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.732482  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:11.738119  9118 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:11.732492  9594 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:11.735092  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:11.735553  9312 tablet_bootstrap.cc:492] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:11.739902  9312 ts_tablet_manager.cc:1403] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.110s	user 0.031s	sys 0.009s
I20260709 13:03:11.723423  9270 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:11.740700  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:03:11.743127  9238 leader_election.cc:343] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.744154  9515 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:11.744603  9515 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.744958  9515 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.759541  9312 raft_consensus.cc:359] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.760385  9312 raft_consensus.cc:385] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.760694  9312 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.761386  9312 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.762611  9515 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.763078  9312 ts_tablet_manager.cc:1434] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.000s
I20260709 13:03:11.780143  9477 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:11.783545  9312 tablet_bootstrap.cc:492] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:11.791501  9477 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.792287  9477 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.797459  9515 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:11.783140  9670 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.799232  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:11.799628  9670 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:11.800875  9235 leader_election.cc:343] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.801297  9120 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:11.797228  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:11.802663  9116 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.803094  9477 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:11.804694  9670 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.806859  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:11.809152  9162 leader_election.cc:343] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.818981  9313 tablet_bootstrap.cc:492] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.819346  9314 tablet_bootstrap.cc:492] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:11.819499  9313 ts_tablet_manager.cc:1403] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.101s	user 0.019s	sys 0.020s
I20260709 13:03:11.823179  9313 raft_consensus.cc:359] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.819959  9314 ts_tablet_manager.cc:1403] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.120s	user 0.026s	sys 0.015s
I20260709 13:03:11.823450  9017 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.830353  9313 raft_consensus.cc:385] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.830457  9270 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:11.830796  9313 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.832168  9701 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:11.833346  9314 raft_consensus.cc:359] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.834331  9312 tablet_bootstrap.cc:654] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.834429  9314 raft_consensus.cc:385] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.834983  9314 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.836609  9314 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.840533  9314 ts_tablet_manager.cc:1434] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:03:11.841743  9314 tablet_bootstrap.cc:492] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:11.831529  9313 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.847409  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:11.848120  9120 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:11.834956  9116 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:11.850133  9314 tablet_bootstrap.cc:654] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.850389  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:11.850973  9701 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:11.852927  9313 ts_tablet_manager.cc:1434] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.005s
I20260709 13:03:11.853667  9477 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:11.854195  9313 tablet_bootstrap.cc:492] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:11.854151  9477 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:11.855011  9477 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.862447  9313 tablet_bootstrap.cc:654] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.867112  9314 tablet_bootstrap.cc:492] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:11.867794  9314 ts_tablet_manager.cc:1403] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.005s
I20260709 13:03:11.870705  9314 raft_consensus.cc:359] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.871515  9314 raft_consensus.cc:385] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.871891  9314 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.872756  9314 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.875000  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:11.877120  9313 tablet_bootstrap.cc:492] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.878983  9313 ts_tablet_manager.cc:1403] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.008s
I20260709 13:03:11.876963  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:11.878471  9314 ts_tablet_manager.cc:1434] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.003s
I20260709 13:03:11.882740  9314 tablet_bootstrap.cc:492] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:11.884579  9312 tablet_bootstrap.cc:492] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:11.885110  9312 ts_tablet_manager.cc:1403] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.104s	user 0.010s	sys 0.006s
I20260709 13:03:11.887631  9312 raft_consensus.cc:359] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.888322  9312 raft_consensus.cc:385] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.886088  9313 raft_consensus.cc:359] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.888585  9312 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.889235  9312 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.891381  9312 ts_tablet_manager.cc:1434] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 13:03:11.892282  9314 tablet_bootstrap.cc:654] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.888803  9313 raft_consensus.cc:385] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.892793  9313 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.892475  9021 catalog_manager.cc:5697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.893687  9313 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.895957  9313 ts_tablet_manager.cc:1434] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.002s
I20260709 13:03:11.896967  9313 tablet_bootstrap.cc:492] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:11.892333  9312 tablet_bootstrap.cc:492] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:11.905452  9477 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.905587  9313 tablet_bootstrap.cc:654] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.906006  9477 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.909885  9477 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:11.910063  9120 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.911619  9312 tablet_bootstrap.cc:654] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.912338  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:11.915671  9235 leader_election.cc:343] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.921468  9313 tablet_bootstrap.cc:492] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.922120  9313 ts_tablet_manager.cc:1403] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.006s
I20260709 13:03:11.925058  9588 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:11.931088  9313 raft_consensus.cc:359] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.931911  9313 raft_consensus.cc:385] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:11.932232  9313 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:11.932868  9313 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.935108  9689 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.935943  9689 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:11.944864  9238 leader_election.cc:343] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.949594  9689 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.952910  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:11.955026  9313 ts_tablet_manager.cc:1434] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.002s
I20260709 13:03:11.956586  9313 tablet_bootstrap.cc:492] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
W20260709 13:03:11.958666  9087 leader_election.cc:343] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:11.960611  9238 leader_election.cc:343] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:11.961067  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:11.962091  9477 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:11.963866  9656 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.964076  9161 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:11.964375  9656 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.966481  9656 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:11.969796  9701 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:11.970237  9701 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:11.970582  9701 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.974440  9313 tablet_bootstrap.cc:654] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:11.977818  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:11.978679  9270 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:11.981459  9701 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:11.986111  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:11.986845  9269 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:11.988722  9313 tablet_bootstrap.cc:492] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:11.989432  9313 ts_tablet_manager.cc:1403] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.005s
I20260709 13:03:11.995952  9269 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:11.999895  9701 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:12.002203  9588 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.002866  9588 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.005878  9588 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:12.004498  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:12.024075  9670 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:12.024653  9670 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:12.025656  9670 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.025995  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:12.027134  9118 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:12.033113  9235 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:12.035807  9313 raft_consensus.cc:359] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.036892  9313 raft_consensus.cc:385] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.037257  9313 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.040464  9594 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:12.040886  9594 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.041249  9594 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.050626  9594 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.070536  9313 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.074110  9594 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:12.074487  9313 ts_tablet_manager.cc:1434] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.085s	user 0.010s	sys 0.000s
I20260709 13:03:12.076692  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:12.076610  9018 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.077529  9118 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.088577  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:12.089996  9118 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:12.091734  9235 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:12.093859  9588 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:12.094928  9588 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:12.096333  9588 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.105265  9313 tablet_bootstrap.cc:492] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:12.120783  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:12.121531  9122 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.123768  9588 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:12.125265  9018 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 2, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:12.145125  9454 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:12.157819  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:12.178174  9122 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:12.189786  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:12.190802  9197 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:12.219194  9313 tablet_bootstrap.cc:654] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.221228  9312 tablet_bootstrap.cc:492] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:12.221835  9312 ts_tablet_manager.cc:1403] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.330s	user 0.033s	sys 0.024s
I20260709 13:03:12.233726  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:12.234479  9195 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:12.265692  9312 raft_consensus.cc:359] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.266574  9312 raft_consensus.cc:385] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.266875  9312 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.267798  9312 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.270258  9312 ts_tablet_manager.cc:1434] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.048s	user 0.015s	sys 0.011s
I20260709 13:03:12.277998  9588 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.278057  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:12.278682  9588 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.281911  9313 tablet_bootstrap.cc:492] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:12.286067  9588 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:12.286358  9195 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:12.286813  9313 ts_tablet_manager.cc:1403] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.211s	user 0.013s	sys 0.002s
I20260709 13:03:12.286144  9192 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.281890  9314 tablet_bootstrap.cc:492] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:12.296253  9314 ts_tablet_manager.cc:1403] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.414s	user 0.152s	sys 0.172s
I20260709 13:03:12.298851  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:12.300330  9312 tablet_bootstrap.cc:492] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:12.305425  9314 raft_consensus.cc:359] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.306167  9314 raft_consensus.cc:385] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.306449  9314 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.309465  9454 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.310014  9454 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.312579  9454 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:12.313133  9689 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:12.313916  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:03:12.315089  9235 leader_election.cc:343] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.316713  9313 raft_consensus.cc:359] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.317509  9313 raft_consensus.cc:385] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.317811  9313 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.318235  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:12.318523  9313 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.319569  9312 tablet_bootstrap.cc:654] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.320474  9314 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.321794  9270 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:12.322834  9314 ts_tablet_manager.cc:1434] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.002s
W20260709 13:03:12.325400  9238 leader_election.cc:343] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.325886  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:12.326637  9588 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:12.330651  9454 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.331187  9454 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.332195  9313 ts_tablet_manager.cc:1434] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.002s
I20260709 13:03:12.333338  9454 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:12.335909  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:12.339247  9089 leader_election.cc:343] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.339775  9089 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:12.340848  9454 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:12.341389  9087 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:12.344580  9454 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:12.344947  9454 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.345295  9454 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.349016  9477 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:03:12.352442  9087 leader_election.cc:343] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.323926  9314 tablet_bootstrap.cc:492] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:12.359723  9314 tablet_bootstrap.cc:654] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.394291  9312 tablet_bootstrap.cc:492] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:12.396935  9312 ts_tablet_manager.cc:1403] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.097s	user 0.032s	sys 0.041s
I20260709 13:03:12.333352  9313 tablet_bootstrap.cc:492] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:12.399308  9314 tablet_bootstrap.cc:492] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:12.399411  9724 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.399837  9314 ts_tablet_manager.cc:1403] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.076s	user 0.010s	sys 0.002s
I20260709 13:03:12.400276  9724 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.402778  9314 raft_consensus.cc:359] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.403503  9314 raft_consensus.cc:385] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.364120  9477 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:03:12.415000  9314 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.367794  9339 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260709 13:03:12.384785  9701 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.389241  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:12.416322  9701 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.417045  9589 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:12.418937  9701 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:12.367007  9454 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.405463  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:12.426517  9269 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:12.428592  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:12.434865  9314 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.435525  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:12.436221  9454 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:12.413676  9312 raft_consensus.cc:359] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.437065  9312 raft_consensus.cc:385] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.437355  9312 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.437999  9312 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.403661  9724 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:12.440690  9312 ts_tablet_manager.cc:1434] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:03:12.440537  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:12.441327  9270 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:12.441771  9312 tablet_bootstrap.cc:492] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:12.436342  9269 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.444437  9689 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:12.444890  9689 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.445268  9689 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.446125  9314 ts_tablet_manager.cc:1434] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:03:12.447252  9314 tablet_bootstrap.cc:492] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Bootstrap starting.
I20260709 13:03:12.410890  9313 tablet_bootstrap.cc:654] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.449705  9269 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:12.450946  9087 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:12.452194  9724 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:12.452749  9724 raft_consensus.cc:697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:12.453738  9724 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.463202  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:12.499430  9314 tablet_bootstrap.cc:654] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:12.528952  9122 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 13:03:12.534786  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 128866us
I20260709 13:03:12.492839  9589 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:03:12.537623  9089 leader_election.cc:343] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.576318  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:12.538838  9589 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:12.538352  9723 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:12.621582  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 79224.4us
I20260709 13:03:12.658555  9312 tablet_bootstrap.cc:654] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:12.631125  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:12.573933  9672 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:03:12.538278  9196 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.623220  9689 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.656548  9589 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:12.683478  9313 tablet_bootstrap.cc:492] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:12.686896  9313 ts_tablet_manager.cc:1403] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.354s	user 0.013s	sys 0.004s
I20260709 13:03:12.654430  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:12.715193  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:12.715888  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:12.717728  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:03:12.726267  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 194187us
W20260709 13:03:12.732511  9089 leader_election.cc:343] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.668134  9723 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.733140  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:12.736284  9454 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:12.739256  9089 leader_election.cc:343] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.736020  9117 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:03:12.767233  9238 leader_election.cc:343] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:12.768491  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:12.779263  9198 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:03:12.793349  9314 tablet_bootstrap.cc:492] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: No bootstrap required, opened a new log
I20260709 13:03:12.794193  9314 ts_tablet_manager.cc:1403] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Time spent bootstrapping tablet: real 0.347s	user 0.019s	sys 0.011s
I20260709 13:03:12.796772  9656 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:03:12.822319  9739 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:12.828154  9730 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.165s	user 0.003s	sys 0.000s
I20260709 13:03:12.816114  9689 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:12.829910  9739 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:12.843294  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:12.875474  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:12.877542  9723 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:12.877311  9313 raft_consensus.cc:359] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.878049  9313 raft_consensus.cc:385] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:12.878315  9313 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:12.881479  9313 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.899925  9588 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.910852  9588 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:12.941926  9313 ts_tablet_manager.cc:1434] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.255s	user 0.007s	sys 0.001s
I20260709 13:03:12.948119  9723 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.000051  9747 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.000936  9747 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.023280  9313 tablet_bootstrap.cc:492] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Bootstrap starting.
I20260709 13:03:12.997785  9588 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:13.028355  9314 raft_consensus.cc:359] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:13.064723  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 202549us
I20260709 13:03:13.006494  9589 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.010785  9723 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.132934  9314 raft_consensus.cc:385] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:13.133911  9314 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Initialized, Role: FOLLOWER
W20260709 13:03:13.139552  9089 leader_election.cc:343] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:13.135099  9314 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.153597  9195 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:13.132468  9701 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:13.170159  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 402951us
I20260709 13:03:13.170984  9314 ts_tablet_manager.cc:1434] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Time spent starting tablet: real 0.376s	user 0.010s	sys 0.024s
I20260709 13:03:13.168504  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:13.194545  9115 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:13.208288  9235 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:13.163849  9701 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.198510  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:13.308137  9759 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:03:13.300343  9118 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:13.390340  9312 tablet_bootstrap.cc:492] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:13.390894  9312 ts_tablet_manager.cc:1403] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.949s	user 0.022s	sys 0.008s
I20260709 13:03:13.402102  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:13.402804  9269 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.414058  9269 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:13.414163  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:13.415098  9272 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:13.406230  9723 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:13.423825  9761 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:13.424363  9761 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.433251  9313 tablet_bootstrap.cc:654] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:13.439972  9312 raft_consensus.cc:359] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.440739  9312 raft_consensus.cc:385] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:13.441042  9312 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:13.441731  9312 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.443295  9312 ts_tablet_manager.cc:1434] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.001s
I20260709 13:03:13.450217  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:13.450951  9193 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:13.428385  9761 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.472494  9761 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.471417  9589 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:13.477006  9761 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:13.480863  9313 tablet_bootstrap.cc:492] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: No bootstrap required, opened a new log
I20260709 13:03:13.481482  9313 ts_tablet_manager.cc:1403] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Time spent bootstrapping tablet: real 0.458s	user 0.021s	sys 0.015s
W20260709 13:03:13.482025  9752 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.289s	user 0.001s	sys 0.010s
I20260709 13:03:13.485319  9313 raft_consensus.cc:359] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.486264  9313 raft_consensus.cc:385] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:13.486577  9313 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Initialized, Role: FOLLOWER
I20260709 13:03:13.487335  9313 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.489333  9313 ts_tablet_manager.cc:1434] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
W20260709 13:03:13.497363  9089 consensus_peers.cc:597] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:13.497634  9312 tablet_bootstrap.cc:492] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:13.497982  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:13.499689  9515 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:13.505052  9753 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:13.515376  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:13.516743  9189 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.522841  9161 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:13.530752  9189 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:13.531699  9755 raft_consensus.cc:2764] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:03:13.539721  9701 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:13.548220  9747 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:03:13.553732  9089 leader_election.cc:343] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:13.560262  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:13.561304  9117 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:13.564185  9739 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.539448  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:13.566960  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:13.564749  9739 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.567612  9272 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:13.577188  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:13.577831  9761 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:13.578385  9761 raft_consensus.cc:697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:13.579249  9761 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.581744  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:13.584177  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:13.596228  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:13.597198  9270 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:13.594589  9672 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.612541  9312 tablet_bootstrap.cc:654] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:13.621599  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 78843.7us
I20260709 13:03:13.623379  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:13.624070  9339 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:13.624557  9339 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:13.626212  9162 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:13.624311  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:13.605430  9672 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.625339  9339 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.634945  9724 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:13.637300  9724 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.637790  9724 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.649506  9743 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:03:13.653442  9739 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:13.634835  9689 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:03:13.677767  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 51631.6us
I20260709 13:03:13.677568  9017 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36610 (request call id 96) took 1101 ms. Trace:
W20260709 13:03:13.678391  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 60249.2us
I20260709 13:03:13.684202  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:13.682649  9734 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:13.695586  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 68252.9us
I20260709 13:03:13.699818  9757 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:13.681031  9689 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.682389  9017 rpcz_store.cc:276] 0709 13:03:12.657129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:12.676648 (+ 19519us) service_pool.cc:224] Handling call
0709 13:03:13.677491 (+1000843us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10174,"reported_tablets":11}
I20260709 13:03:13.720054  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:03:13.733529  9238 leader_election.cc:343] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:13.746385  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:13.750861  9672 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:13.768181  9689 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:03:13.772907  9069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:13.775003  9312 tablet_bootstrap.cc:492] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:13.775637  9312 ts_tablet_manager.cc:1403] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.278s	user 0.022s	sys 0.002s
I20260709 13:03:13.803212  9780 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:13.803898  9780 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.806493  9312 raft_consensus.cc:359] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.808017  9312 raft_consensus.cc:385] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:13.808342  9312 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:13.813766  9589 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.814291  9589 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.814551  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:13.818015  9589 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:13.817076  9312 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.820587  9312 ts_tablet_manager.cc:1434] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.001s
W20260709 13:03:13.825799  9089 leader_election.cc:343] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:13.832099  9780 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:13.835714  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:13.838052  9267 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 0.
I20260709 13:03:13.841954  9162 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:13.843796  9734 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:13.844204  9734 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.844514  9734 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.850709  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:13.857259  9734 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.865576  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:13.866503  9734 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:03:13.867151  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 164567us
I20260709 13:03:13.867508  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:13.868178  9267 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.836738  9724 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.869117  9117 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:13.870151  9724 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:13.872145  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:13.873119  9778 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:13.873533  9778 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.873886  9778 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.877329  9267 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:03:13.878679  9162 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:13.880748  9731 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:13.881425  9731 raft_consensus.cc:697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:13.880784  9017 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:13.881615  9778 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.882390  9731 consensus_queue.cc:237] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.885320  9787 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:13.897197  9017 catalog_manager.cc:5697] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:13.907611  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:13.910503  9114 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.906553  9778 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:13.917532  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:13.918490  9117 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:13.925099  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:13.927239  9312 tablet_bootstrap.cc:492] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:13.931348  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:13.932008  9198 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.931378  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:13.936546  9115 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 0.
I20260709 13:03:13.938686  9114 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:13.943975  9198 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:13.946851  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:13.947142  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:13.947638  9267 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 0.
I20260709 13:03:13.947877  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:13.952050  9089 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:13.953169  9725 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:13.953727  9725 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:13.954641  9725 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.961812  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:13.962918  9751 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:13.963294  9751 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.963608  9751 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.965420  9087 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:13.965581  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:13.966246  9778 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:13.966358  9271 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 1: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:13.967300  9312 tablet_bootstrap.cc:654] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:13.967494  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:13.968663  9087 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:13.969460  9778 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:13.969914  9778 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:13.970275  9778 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.973665  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:13.974848  9265 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:13.977921  9751 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.978050  9778 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:13.980700  9751 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:13.981318  9778 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:13.982988  9265 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:13.972889  9724 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:13.984282  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:03:13.985687  9089 leader_election.cc:343] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:13.984931  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:13.987869  9786 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:13.989104  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:13.990753  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:13.991442  9267 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.000273  9267 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
W20260709 13:03:13.990898  9238 leader_election.cc:343] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:14.002460  9312 tablet_bootstrap.cc:492] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:14.003962  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:14.004215  9312 ts_tablet_manager.cc:1403] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.077s	user 0.017s	sys 0.000s
I20260709 13:03:14.004688  9114 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.007915  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:14.010624  9725 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.013813  9312 raft_consensus.cc:359] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.014668  9312 raft_consensus.cc:385] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.014935  9312 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.015612  9312 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.014098  9725 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:14.018232  9312 ts_tablet_manager.cc:1434] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:03:14.019271  9312 tablet_bootstrap.cc:492] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:14.018939  9725 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.021095  9114 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:14.022312  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:14.023444  9741 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:14.025108  9741 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:14.025981  9741 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.028162  9312 tablet_bootstrap.cc:654] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.039005  9312 tablet_bootstrap.cc:492] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:14.039474  9312 ts_tablet_manager.cc:1403] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.003s
I20260709 13:03:14.042150  9312 raft_consensus.cc:359] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.045399  9312 raft_consensus.cc:385] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.045765  9312 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.046407  9312 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.048009  9312 ts_tablet_manager.cc:1434] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 13:03:14.048792  9312 tablet_bootstrap.cc:492] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466: Bootstrap starting.
I20260709 13:03:14.049412  9020 catalog_manager.cc:5697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:14.053542  9312 tablet_bootstrap.cc:654] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:14.057721  9734 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.058365  9734 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.062906  9734 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:14.064646  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:14.065577  9114 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:14.067288  9312 tablet_bootstrap.cc:492] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466: No bootstrap required, opened a new log
I20260709 13:03:14.068400  9312 ts_tablet_manager.cc:1403] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.001s
I20260709 13:03:14.065099  9021 catalog_manager.cc:5697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.072086  9312 raft_consensus.cc:359] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.072804  9312 raft_consensus.cc:385] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:14.073096  9312 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Initialized, Role: FOLLOWER
I20260709 13:03:14.073724  9312 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.071652  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:14.075898  9312 ts_tablet_manager.cc:1434] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:03:14.076802  9162 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:14.077702  9734 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.093544  9734 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:14.093290  9017 catalog_manager.cc:5697] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.095140  9017 catalog_manager.cc:5697] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.111588  9020 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:14.138826  9194 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.153743  9741 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:14.167078  9761 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:14.182547  9725 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:14.196857  9778 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:14.304991  9782 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:14.315915  9782 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:14.356077  9782 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:14.356535  9782 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:14.358289  9782 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:14.359069  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:14.359333  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:14.360100  9271 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 1: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:14.361029  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:14.361761  9782 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:14.367871  9782 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:14.420032  9724 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:14.434402  9778 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:14.468333  9751 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:14.479385  9588 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:14.504920  9778 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:14.528559  9787 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:15.609344  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:15.638391  9817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:15.638609  9816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:15.640260  8021 server_base.cc:1061] running on GCE node
W20260709 13:03:15.640133  9819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:15.641872  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:15.642181  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:15.642449  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602195642406 us; error 0 us; skew 500 ppm
I20260709 13:03:15.643072  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:15.646322  8021 webserver.cc:533] Webserver started at http://127.7.213.68:33961/ using document root <none> and password file <none>
I20260709 13:03:15.646844  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:15.647058  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:15.647378  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:15.648619  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-3-root/instance:
uuid: "92919b70789041c8a0de0b3525154fd1"
format_stamp: "Formatted at 2026-07-09 13:03:15 on dist-test-slave-9g29"
I20260709 13:03:15.654111  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:03:15.658279  9824 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:15.659301  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:03:15.659786  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-3-root
uuid: "92919b70789041c8a0de0b3525154fd1"
format_stamp: "Formatted at 2026-07-09 13:03:15 on dist-test-slave-9g29"
I20260709 13:03:15.660140  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:15.684731  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:15.686434  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:15.688165  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:15.690763  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:15.690963  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:15.691180  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:15.691323  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:15.779253  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.68:43523
I20260709 13:03:15.779409  9887 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.68:43523 every 8 connection(s)
I20260709 13:03:15.783296  8021 auto_leader_rebalancer-test.cc:384] add a tserver: 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:15.804991  9888 heartbeater.cc:344] Connected to a master server at 127.7.213.126:45141
I20260709 13:03:15.805495  9888 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:15.806309  9888 heartbeater.cc:507] Master 127.7.213.126:45141 requested a full tablet report, sending...
I20260709 13:03:15.808326  9020 ts_manager.cc:194] Registered new tserver with Master: 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:15.809808  9020 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42488
I20260709 13:03:15.935503  9114 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:15.942833  9271 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:15.944440  9815 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:15.944674  9194 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:15.946688  9724 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:15.963974  9089 consensus_peers.cc:597] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a6c1cc32069145ebbbacf127d2516fe8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.965022  9724 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:15.968513  9189 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:15.975411  9271 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:15.982586  9020 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.029320  9189 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.040297  9271 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:16.041746  9884 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.042624  9114 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:16.044351  9731 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.058305  9782 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:03:16.060689  9164 consensus_peers.cc:597] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aaf63fb358d848ad991a929b2e8e0168. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:16.059818  9271 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.076467  9114 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.078029  9021 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:16.104017  9271 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.124411  9189 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:16.125772  9114 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:16.127698  9912 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:16.132618  9903 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:03:16.140573  9238 consensus_peers.cc:597] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b7da62e1fea4b79963197feee655d26. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:16.146566  9903 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.147933  9114 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.151799  9189 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.163781  9017 catalog_manager.cc:5697] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.160076  9114 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.173220  9267 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:16.173274  9089 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 29257e9d0bc4408583c5df86d6e2fd64. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:16.175436  9189 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:16.176178  9778 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:16.180845  9815 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:16.197885  9271 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.196193  9815 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.210901  9189 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.231514  9020 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:16.374650  9919 ts_tablet_manager.cc:933] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:16.376762  9919 tablet_copy_client.cc:323] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:16.445972  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet a6c1cc32069145ebbbacf127d2516fe8 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:16.446632  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet a6c1cc32069145ebbbacf127d2516fe8 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-a6c1cc32069145ebbbacf127d2516fe8
I20260709 13:03:16.453467  9133 tablet_copy_source_session.cc:215] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:16.457283  9919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c1cc32069145ebbbacf127d2516fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:16.475610  9919 tablet_copy_client.cc:806] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:16.476517  9919 tablet_copy_client.cc:670] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:16.482323  9919 tablet_copy_client.cc:538] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:16.496882  9919 tablet_bootstrap.cc:492] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:16.520219  9919 tablet_bootstrap.cc:492] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:16.521270  9919 tablet_bootstrap.cc:492] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:16.521951  9919 ts_tablet_manager.cc:1403] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.025s	user 0.021s	sys 0.000s
I20260709 13:03:16.525118  9919 raft_consensus.cc:359] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.525842  9919 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:16.526440  9919 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.529696  9888 heartbeater.cc:499] Master 127.7.213.126:45141 was elected leader, sending a full tablet report...
I20260709 13:03:16.533363  9919 ts_tablet_manager.cc:1434] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.007s
I20260709 13:03:16.537297  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-a6c1cc32069145ebbbacf127d2516fe8 received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:16.537865  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-a6c1cc32069145ebbbacf127d2516fe8 on tablet a6c1cc32069145ebbbacf127d2516fe8 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:16.602421  9919 ts_tablet_manager.cc:933] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:16.604035  9919 tablet_copy_client.cc:323] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:16.617988  9208 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet aaf63fb358d848ad991a929b2e8e0168 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:16.618621  9208 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet aaf63fb358d848ad991a929b2e8e0168 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-aaf63fb358d848ad991a929b2e8e0168
I20260709 13:03:16.626387  9208 tablet_copy_source_session.cc:215] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:16.629869  9919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaf63fb358d848ad991a929b2e8e0168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:16.672480  9919 tablet_copy_client.cc:806] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:16.673117  9919 tablet_copy_client.cc:670] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:16.678227  9919 tablet_copy_client.cc:538] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:16.688668  9919 tablet_bootstrap.cc:492] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:16.711112  9919 tablet_bootstrap.cc:492] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:16.712160  9919 tablet_bootstrap.cc:492] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:16.712895  9919 ts_tablet_manager.cc:1403] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.007s
I20260709 13:03:16.715943  9919 raft_consensus.cc:359] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.716894  9919 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:16.717645  9919 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.722007  9919 ts_tablet_manager.cc:1434] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:03:16.727152  9208 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-aaf63fb358d848ad991a929b2e8e0168 received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:16.727702  9208 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-aaf63fb358d848ad991a929b2e8e0168 on tablet aaf63fb358d848ad991a929b2e8e0168 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:16.741582  9919 ts_tablet_manager.cc:933] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:16.743929  9919 tablet_copy_client.cc:323] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:16.758167  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 2b7da62e1fea4b79963197feee655d26 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:16.758797  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 2b7da62e1fea4b79963197feee655d26 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-2b7da62e1fea4b79963197feee655d26
I20260709 13:03:16.767856  9282 tablet_copy_source_session.cc:215] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:16.769148  9929 ts_tablet_manager.cc:933] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:16.771183  9929 tablet_copy_client.cc:323] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:16.772922  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet 29257e9d0bc4408583c5df86d6e2fd64 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:16.773458  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet 29257e9d0bc4408583c5df86d6e2fd64 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-29257e9d0bc4408583c5df86d6e2fd64
I20260709 13:03:16.781561  9919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b7da62e1fea4b79963197feee655d26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:16.780326  9133 tablet_copy_source_session.cc:215] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:16.786868  9929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29257e9d0bc4408583c5df86d6e2fd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:16.810549  9919 tablet_copy_client.cc:806] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:16.811812  9919 tablet_copy_client.cc:670] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:16.826117  9919 tablet_copy_client.cc:538] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:16.834539  9929 tablet_copy_client.cc:806] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:16.838649  9929 tablet_copy_client.cc:670] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:16.839718  9919 tablet_bootstrap.cc:492] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:16.842789  9929 tablet_copy_client.cc:538] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:16.860405  9929 tablet_bootstrap.cc:492] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:16.861577  9919 tablet_bootstrap.cc:492] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:16.862664  9919 tablet_bootstrap.cc:492] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:16.863336  9919 ts_tablet_manager.cc:1403] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.004s
I20260709 13:03:16.866245  9919 raft_consensus.cc:359] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.867146  9919 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:16.867846  9919 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.884326  9919 ts_tablet_manager.cc:1434] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:03:16.886611  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-2b7da62e1fea4b79963197feee655d26 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:16.887068  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-2b7da62e1fea4b79963197feee655d26 on tablet 2b7da62e1fea4b79963197feee655d26 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:16.895191  9929 tablet_bootstrap.cc:492] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:16.896117  9929 tablet_bootstrap.cc:492] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:16.896780  9929 ts_tablet_manager.cc:1403] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.011s
I20260709 13:03:16.899343  9929 raft_consensus.cc:359] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.900230  9929 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:16.900936  9929 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:16.903942  9929 ts_tablet_manager.cc:1434] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:16.905627  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-29257e9d0bc4408583c5df86d6e2fd64 received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:16.906080  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-29257e9d0bc4408583c5df86d6e2fd64 on tablet 29257e9d0bc4408583c5df86d6e2fd64 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:16.966351  9862 raft_consensus.cc:1217] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:17.058718  9862 raft_consensus.cc:1217] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:17.186327  9862 raft_consensus.cc:1217] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:17.218811  9862 raft_consensus.cc:1217] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:17.426862  9778 raft_consensus.cc:1064] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:17.429126  9778 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:17.436721  9862 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.439872  9265 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.441395  9935 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.442823  9778 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.450903  9189 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.456303  9778 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.457950  9931 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.460573  9265 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.476420  9862 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.505760  9021 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.515935  9189 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.676951  9934 raft_consensus.cc:1064] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:17.679013  9934 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:17.685927  9265 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.686367  9862 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.687613  9114 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.688714  9884 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.690135  9947 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.691052  9782 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.702643  9903 raft_consensus.cc:1064] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:17.705739  9903 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:17.721266  9931 raft_consensus.cc:1064] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:17.727344  9931 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:17.744661  9782 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.746290  9862 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.759032  9861 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.759704  9860 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.761255  9935 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:17.761770  9265 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.764418  9903 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.766969  9271 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.769851  9931 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.772454  9121 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.774920  9741 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.785918  9198 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.787377  9194 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:17.787720  9021 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:17.789732  9935 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:03:17.797842  9912 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:17.825112  9114 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.841836  9191 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:17.850481  9862 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:17.853405  9271 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:17.855970  9884 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:17.865979  9815 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.870708  9860 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.892498  9947 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:17.893010  9020 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.901440  9934 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.905436  9861 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.909497  9271 raft_consensus.cc:2955] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.912808  9860 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.911238  9903 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.932974  9265 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.943917  9017 catalog_manager.cc:5697] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.945071  9198 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.947247  9194 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:17.986922  9121 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.000399  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet aaf63fb358d848ad991a929b2e8e0168 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:18.016953  9114 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:18.017563  9020 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.030113  9198 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:18.030964  9861 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:18.031689  9815 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:18.034303  9778 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:18.041086  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet aaf63fb358d848ad991a929b2e8e0168 with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:18.048887  9935 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.055554  9861 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.060698  9194 raft_consensus.cc:2955] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.062858  9114 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:18.072433  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a6c1cc32069145ebbbacf127d2516fe8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:18.090443  9020 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.091658  9861 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:18.091917  9267 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:18.093245  9194 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:18.095057  9962 tablet_replica.cc:333] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:18.093302  9940 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:03:18.096150  9962 raft_consensus.cc:2243] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:18.096730  9935 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:18.097059  9962 raft_consensus.cc:2272] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:18.107775  9962 ts_tablet_manager.cc:1916] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:18.111820  9815 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.113016  9861 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.117929  9114 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:18.120735  9741 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:18.124010  9860 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:18.124325  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet a6c1cc32069145ebbbacf127d2516fe8 with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:18.128412  9963 tablet_replica.cc:333] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:18.130043  9963 raft_consensus.cc:2243] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:18.130859  9963 raft_consensus.cc:2272] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:18.130340  9948 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.132714  9121 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.136255  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 29257e9d0bc4408583c5df86d6e2fd64 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:18.145525  9962 ts_tablet_manager.cc:1929] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:18.146160  9962 log.cc:1199] T aaf63fb358d848ad991a929b2e8e0168 P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/aaf63fb358d848ad991a929b2e8e0168
I20260709 13:03:18.146790  9963 ts_tablet_manager.cc:1916] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:18.129510  9194 raft_consensus.cc:2955] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.153517  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet aaf63fb358d848ad991a929b2e8e0168 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:18.153889  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2b7da62e1fea4b79963197feee655d26 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:18.157332  9017 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.157997  9018 catalog_manager.cc:5697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:18.160907  9948 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:18.177992  9963 ts_tablet_manager.cc:1929] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:18.179509  9963 log.cc:1199] T a6c1cc32069145ebbbacf127d2516fe8 P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/a6c1cc32069145ebbbacf127d2516fe8
I20260709 13:03:18.180619  9861 raft_consensus.cc:2955] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:18.184795  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 29257e9d0bc4408583c5df86d6e2fd64 with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:18.186949  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet a6c1cc32069145ebbbacf127d2516fe8 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:18.185818  9963 tablet_replica.cc:333] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:18.197485  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet 2b7da62e1fea4b79963197feee655d26 with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:18.239679  9964 tablet_replica.cc:333] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:18.240762  9964 raft_consensus.cc:2243] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:18.241547  9964 raft_consensus.cc:2272] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:18.250881  9964 ts_tablet_manager.cc:1916] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:18.266672  9963 raft_consensus.cc:2243] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:18.267496  9963 raft_consensus.cc:2272] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:18.271153  9963 ts_tablet_manager.cc:1916] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:18.298152  9964 ts_tablet_manager.cc:1929] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:18.298295  9963 ts_tablet_manager.cc:1929] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:18.298676  9964 log.cc:1199] T 2b7da62e1fea4b79963197feee655d26 P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/2b7da62e1fea4b79963197feee655d26
I20260709 13:03:18.298759  9963 log.cc:1199] T 29257e9d0bc4408583c5df86d6e2fd64 P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/29257e9d0bc4408583c5df86d6e2fd64
I20260709 13:03:18.300769  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 29257e9d0bc4408583c5df86d6e2fd64 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:18.300882  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet 2b7da62e1fea4b79963197feee655d26 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:19.293483  9114 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:19.305215  9089 consensus_peers.cc:597] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 00ff82ce7b7342a4b052db43182f4a99. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:19.305274  9267 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:19.307339  9815 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:19.307822  9195 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:19.309360  9815 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:19.319315  9969 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.321616  9195 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.322455  9267 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.336628  9195 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:19.340073  9017 catalog_manager.cc:5697] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:03:19.365819  9164 consensus_peers.cc:597] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f84c9fc901364e60b2c414ee3b7a7e65. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:19.367786  9267 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:19.368739  9114 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:19.369499  9782 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:19.370631  9970 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:19.394908  9971 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.397476  9114 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.427143  9271 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:19.444844  9020 catalog_manager.cc:5697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:19.458860  9267 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:03:19.462688  9238 consensus_peers.cc:597] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbfd5d6417134672a62b2757b6d55dff. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:19.462531  9195 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:19.464409  9114 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:19.464840  9903 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:19.466219  9973 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:19.494871  9903 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.498108  9195 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.504575  9114 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.513902  9271 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:19.517354  9018 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.533485  9195 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:19.533596  9238 consensus_peers.cc:597] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9cbe8a4b1d764261adeeb8d882fa37e5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:19.536496  9973 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:19.537395  9114 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:19.540911  9973 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:19.549255  9972 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.551690  9195 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.555258  9115 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:19.568594  9018 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:19.758602  9991 ts_tablet_manager.cc:933] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:19.760826  9991 tablet_copy_client.cc:323] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:19.763020  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet 00ff82ce7b7342a4b052db43182f4a99 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:19.763599  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet 00ff82ce7b7342a4b052db43182f4a99 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-00ff82ce7b7342a4b052db43182f4a99
I20260709 13:03:19.771073  9133 tablet_copy_source_session.cc:215] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:19.774688  9991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00ff82ce7b7342a4b052db43182f4a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:19.793642  9991 tablet_copy_client.cc:806] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:19.794730  9991 tablet_copy_client.cc:670] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:19.799230  9991 tablet_copy_client.cc:538] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:19.809973  9991 tablet_bootstrap.cc:492] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:19.818447  9993 ts_tablet_manager.cc:933] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:19.829447  9993 tablet_copy_client.cc:323] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:19.831976  9208 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet f84c9fc901364e60b2c414ee3b7a7e65 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:19.832674  9208 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet f84c9fc901364e60b2c414ee3b7a7e65 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-f84c9fc901364e60b2c414ee3b7a7e65
I20260709 13:03:19.842448  9208 tablet_copy_source_session.cc:215] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:19.843034  9991 tablet_bootstrap.cc:492] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:19.844295  9991 tablet_bootstrap.cc:492] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:19.845101  9991 ts_tablet_manager.cc:1403] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.035s	user 0.020s	sys 0.012s
I20260709 13:03:19.847044  9993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f84c9fc901364e60b2c414ee3b7a7e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:19.849656  9991 raft_consensus.cc:359] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:19.850956  9991 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:19.851764  9991 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:19.873369  9993 tablet_copy_client.cc:806] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:19.874388  9993 tablet_copy_client.cc:670] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:19.876596  9991 ts_tablet_manager.cc:1434] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.031s	user 0.013s	sys 0.017s
I20260709 13:03:19.878968  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-00ff82ce7b7342a4b052db43182f4a99 received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:19.879473  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-00ff82ce7b7342a4b052db43182f4a99 on tablet 00ff82ce7b7342a4b052db43182f4a99 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:19.882329  9993 tablet_copy_client.cc:538] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:19.891023  9993 tablet_bootstrap.cc:492] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:19.912351  9993 tablet_bootstrap.cc:492] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:19.913340  9993 tablet_bootstrap.cc:492] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:19.914124  9993 ts_tablet_manager.cc:1403] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.023s	user 0.023s	sys 0.001s
I20260709 13:03:19.917122  9993 raft_consensus.cc:359] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:19.917992  9993 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:19.918730  9993 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:19.919390  9991 ts_tablet_manager.cc:933] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:19.921303  9993 ts_tablet_manager.cc:1434] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:03:19.923663  9208 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-f84c9fc901364e60b2c414ee3b7a7e65 received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:19.924309  9208 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-f84c9fc901364e60b2c414ee3b7a7e65 on tablet f84c9fc901364e60b2c414ee3b7a7e65 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:19.931983  9991 tablet_copy_client.cc:323] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:19.934386  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 9cbe8a4b1d764261adeeb8d882fa37e5 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:19.934931  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 9cbe8a4b1d764261adeeb8d882fa37e5 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-9cbe8a4b1d764261adeeb8d882fa37e5
I20260709 13:03:19.941684  9282 tablet_copy_source_session.cc:215] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:19.945367  9991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cbe8a4b1d764261adeeb8d882fa37e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:19.947563  9993 ts_tablet_manager.cc:933] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:19.950734  9993 tablet_copy_client.cc:323] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:19.953509  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet bbfd5d6417134672a62b2757b6d55dff from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:19.954293  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet bbfd5d6417134672a62b2757b6d55dff from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-bbfd5d6417134672a62b2757b6d55dff
I20260709 13:03:19.971697  9282 tablet_copy_source_session.cc:215] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:19.973414  9991 tablet_copy_client.cc:806] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:19.974076  9991 tablet_copy_client.cc:670] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:19.975205  9993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbfd5d6417134672a62b2757b6d55dff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:19.978371  9991 tablet_copy_client.cc:538] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:19.993310  9991 tablet_bootstrap.cc:492] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:20.007664  9993 tablet_copy_client.cc:806] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:20.008423  9993 tablet_copy_client.cc:670] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:20.013398  9993 tablet_copy_client.cc:538] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:20.026836  9993 tablet_bootstrap.cc:492] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:20.056253  9993 tablet_bootstrap.cc:492] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:20.059401  9993 tablet_bootstrap.cc:492] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:20.060276  9993 ts_tablet_manager.cc:1403] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.034s	user 0.025s	sys 0.000s
I20260709 13:03:20.060962  9991 tablet_bootstrap.cc:492] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:20.063818  9991 tablet_bootstrap.cc:492] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:20.064810  9991 ts_tablet_manager.cc:1403] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.072s	user 0.021s	sys 0.008s
I20260709 13:03:20.065529  9993 raft_consensus.cc:359] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:20.066898  9993 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:20.067658  9993 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:20.075194  9993 ts_tablet_manager.cc:1434] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.004s
I20260709 13:03:20.069183  9991 raft_consensus.cc:359] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:20.076135  9991 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:20.077209  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-bbfd5d6417134672a62b2757b6d55dff received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:20.077751  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-bbfd5d6417134672a62b2757b6d55dff on tablet bbfd5d6417134672a62b2757b6d55dff with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:20.077507  9991 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:20.080953  9991 ts_tablet_manager.cc:1434] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.016s	user 0.001s	sys 0.006s
I20260709 13:03:20.083869  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-9cbe8a4b1d764261adeeb8d882fa37e5 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:20.084321  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-9cbe8a4b1d764261adeeb8d882fa37e5 on tablet 9cbe8a4b1d764261adeeb8d882fa37e5 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:20.322077  9861 raft_consensus.cc:1217] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:20.374063  9861 raft_consensus.cc:1217] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:20.507414  9861 raft_consensus.cc:1217] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:20.544484  9861 raft_consensus.cc:1217] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:03:20.756878 10003 raft_consensus.cc:1064] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:20.760033 10003 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:20.773171  9265 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:20.773064  9861 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:20.774395  9198 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:20.775321 10003 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:20.776460  9815 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.777374  9935 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:20.803926 10002 raft_consensus.cc:1064] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:20.809360 10002 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:20.816495 10003 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:20.818799  9861 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:20.846294  9017 catalog_manager.cc:5697] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:20.850891  9271 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:20.857722  9198 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:20.900897  9861 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:20.901010  9121 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:20.902635  9971 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.903684  9984 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.907472  9271 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:20.909061  9971 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:20.929330  9984 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:20.932433  9265 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:20.935864  9861 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:20.971827  9973 raft_consensus.cc:1064] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:20.978133  9121 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:20.982035  9973 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:20.984637  9972 raft_consensus.cc:1064] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:20.994408  9972 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:21.019003  9021 catalog_manager.cc:5697] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.037273  9121 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:21.045843  9861 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:21.047410 10005 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:21.052577  9115 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:21.057853  9198 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:21.062724  9115 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:21.070019  9972 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:21.069602  9935 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.069602  9860 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:21.072873  9198 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:21.073871  9861 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:21.075038  9265 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:21.075155  9193 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:21.076437  9267 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:21.077451  9903 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:21.078665  9971 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:21.080314 10002 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:21.087108 10019 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:21.090049 10019 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:21.094511  9861 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:21.099823  9860 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.101050  9861 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.095292 10002 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.109519  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 00ff82ce7b7342a4b052db43182f4a99 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:21.111883 10019 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:21.112538  9271 raft_consensus.cc:2955] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.119293  9903 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:21.143342  9935 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:21.143044  9973 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.168316  9198 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.179533 10019 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.187516  9020 catalog_manager.cc:5697] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.192592  9859 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:03:21.194939  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47416.5us
I20260709 13:03:21.197854  9018 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.207062  9859 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.212818  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet 00ff82ce7b7342a4b052db43182f4a99 with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:21.217195  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f84c9fc901364e60b2c414ee3b7a7e65 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:21.220089  9122 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.235076  9121 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.242816  9018 catalog_manager.cc:5697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:21.253877  9267 raft_consensus.cc:2955] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.261216  9193 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.278288  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet f84c9fc901364e60b2c414ee3b7a7e65 with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:21.282507 10031 tablet_replica.cc:333] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:21.284471 10031 raft_consensus.cc:2243] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.285194 10031 raft_consensus.cc:2272] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.289837 10031 ts_tablet_manager.cc:1916] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:21.290300  9017 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.310348 10030 tablet_replica.cc:333] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:21.324110  9270 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:21.326802 10030 raft_consensus.cc:2243] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.327706 10030 raft_consensus.cc:2272] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.330848  9267 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:21.332340 10030 ts_tablet_manager.cc:1916] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:21.332677  9859 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:21.335160  9973 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:21.339852  9121 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:21.349306  9198 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:21.351678 10031 ts_tablet_manager.cc:1929] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:21.356624  9973 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:21.357085 10031 log.cc:1199] T f84c9fc901364e60b2c414ee3b7a7e65 P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/f84c9fc901364e60b2c414ee3b7a7e65
I20260709 13:03:21.353740 10029 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:21.359337  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet f84c9fc901364e60b2c414ee3b7a7e65 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:21.355257 10019 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.351955  9859 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:21.366495  9859 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.368649 10030 ts_tablet_manager.cc:1929] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:21.369295 10030 log.cc:1199] T 00ff82ce7b7342a4b052db43182f4a99 P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/00ff82ce7b7342a4b052db43182f4a99
I20260709 13:03:21.371622 10017 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:21.373718  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet 00ff82ce7b7342a4b052db43182f4a99 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:21.377136  9972 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.378831  9198 raft_consensus.cc:2955] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.385807  9121 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.391772  9860 raft_consensus.cc:2955] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.403877  9017 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.409701  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bbfd5d6417134672a62b2757b6d55dff with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:21.418880  9020 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:21.436339  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9cbe8a4b1d764261adeeb8d882fa37e5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:21.445637  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet bbfd5d6417134672a62b2757b6d55dff with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:21.446636 10031 tablet_replica.cc:333] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:21.447429  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet 9cbe8a4b1d764261adeeb8d882fa37e5 with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:21.447811 10031 raft_consensus.cc:2243] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.448717 10031 raft_consensus.cc:2272] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.449865 10030 tablet_replica.cc:333] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:21.451035 10030 raft_consensus.cc:2243] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:21.451885 10030 raft_consensus.cc:2272] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:21.459880 10030 ts_tablet_manager.cc:1916] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:21.464196 10031 ts_tablet_manager.cc:1916] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:21.476508 10030 ts_tablet_manager.cc:1929] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:21.477002 10030 log.cc:1199] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/9cbe8a4b1d764261adeeb8d882fa37e5
I20260709 13:03:21.478633  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet 9cbe8a4b1d764261adeeb8d882fa37e5 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:21.481986 10031 ts_tablet_manager.cc:1929] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:03:21.482498 10031 log.cc:1199] T bbfd5d6417134672a62b2757b6d55dff P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/bbfd5d6417134672a62b2757b6d55dff
I20260709 13:03:21.484191  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet bbfd5d6417134672a62b2757b6d55dff (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:22.513329  9115 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:22.519167  9270 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:22.519352  9193 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:22.519625  9089 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43e874ff0724409b88c5ec377b73a06a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:22.520833 10033 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:22.521818 10005 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:22.531071 10005 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.532859  9267 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.535071  9193 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.567525  9017 catalog_manager.cc:5697] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.568833  9267 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:22.591516  9238 consensus_peers.cc:597] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: edfac5554fb144c3ab8a6248c7ea68d8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:22.594071  9198 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:22.606312  9115 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:22.607873  9903 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:03:22.615355 10034 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:22.626798 10034 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.629868  9115 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.640709  9193 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.672436  9021 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.689851  9115 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:22.704525  9191 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:22.705065  9089 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a35f9598c50046539a7f41287876135d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:22.706527  9271 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:22.706429 10048 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:22.707844 10005 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:22.716671 10048 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.720412  9271 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.725345  9193 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.732645  9270 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:22.734157  9017 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:22.749505  9115 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:22.753067  9903 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:22.751001  9238 consensus_peers.cc:597] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0f89ac8f7b142239d6103bd6ed96037. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:22.750288  9191 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:22.757083  9903 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:22.763139 10034 raft_consensus.cc:2955] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.772835  9193 raft_consensus.cc:2955] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.783041  9021 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:22.772931  9115 raft_consensus.cc:2955] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:22.925769 10055 ts_tablet_manager.cc:933] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:22.928711 10055 tablet_copy_client.cc:323] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:22.930394  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet 43e874ff0724409b88c5ec377b73a06a from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:22.930912  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet 43e874ff0724409b88c5ec377b73a06a from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-43e874ff0724409b88c5ec377b73a06a
I20260709 13:03:22.937901  9133 tablet_copy_source_session.cc:215] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:22.941431 10055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e874ff0724409b88c5ec377b73a06a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:22.966097 10055 tablet_copy_client.cc:806] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:22.966847 10055 tablet_copy_client.cc:670] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:22.970911 10055 tablet_copy_client.cc:538] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:22.978811 10055 tablet_bootstrap.cc:492] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:22.999033 10055 tablet_bootstrap.cc:492] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:23.000286 10055 tablet_bootstrap.cc:492] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:23.004598 10055 ts_tablet_manager.cc:1403] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.010s
I20260709 13:03:23.008397 10055 raft_consensus.cc:359] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:23.009480 10055 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:23.010339 10055 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:23.021900 10055 ts_tablet_manager.cc:1434] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.008s
I20260709 13:03:23.024047  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-43e874ff0724409b88c5ec377b73a06a received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:23.024562  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-43e874ff0724409b88c5ec377b73a06a on tablet 43e874ff0724409b88c5ec377b73a06a with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:23.196385 10055 ts_tablet_manager.cc:933] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:23.198096 10055 tablet_copy_client.cc:323] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:23.199559  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet edfac5554fb144c3ab8a6248c7ea68d8 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:23.200098  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet edfac5554fb144c3ab8a6248c7ea68d8 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-edfac5554fb144c3ab8a6248c7ea68d8
I20260709 13:03:23.207683  9282 tablet_copy_source_session.cc:215] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:23.213932 10060 ts_tablet_manager.cc:933] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:23.215415 10055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edfac5554fb144c3ab8a6248c7ea68d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:23.243376 10060 tablet_copy_client.cc:323] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:23.245121  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet c0f89ac8f7b142239d6103bd6ed96037 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:23.245671  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet c0f89ac8f7b142239d6103bd6ed96037 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-c0f89ac8f7b142239d6103bd6ed96037
I20260709 13:03:23.246812 10055 tablet_copy_client.cc:806] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:23.247364 10055 tablet_copy_client.cc:670] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:23.252269 10055 tablet_copy_client.cc:538] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:23.261147  9282 tablet_copy_source_session.cc:215] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:23.293977 10055 tablet_bootstrap.cc:492] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:23.297327 10060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f89ac8f7b142239d6103bd6ed96037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:23.325506 10062 ts_tablet_manager.cc:933] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:23.348117 10062 tablet_copy_client.cc:323] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:23.349761  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet a35f9598c50046539a7f41287876135d from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:23.350291  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet a35f9598c50046539a7f41287876135d from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-a35f9598c50046539a7f41287876135d
I20260709 13:03:23.351719 10060 tablet_copy_client.cc:806] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:23.352778 10060 tablet_copy_client.cc:670] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:23.357293 10060 tablet_copy_client.cc:538] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:23.363006  9133 tablet_copy_source_session.cc:215] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:23.366997 10062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a35f9598c50046539a7f41287876135d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:23.371603 10055 tablet_bootstrap.cc:492] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:23.372690 10055 tablet_bootstrap.cc:492] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:23.373456 10055 ts_tablet_manager.cc:1403] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.080s	user 0.027s	sys 0.015s
I20260709 13:03:23.376518 10055 raft_consensus.cc:359] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:23.379060 10055 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:23.380839 10055 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:23.383404 10055 ts_tablet_manager.cc:1434] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:03:23.385205  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-edfac5554fb144c3ab8a6248c7ea68d8 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:23.385671  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-edfac5554fb144c3ab8a6248c7ea68d8 on tablet edfac5554fb144c3ab8a6248c7ea68d8 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:23.388912 10060 tablet_bootstrap.cc:492] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:23.395277 10062 tablet_copy_client.cc:806] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:23.396246 10062 tablet_copy_client.cc:670] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:23.401367 10062 tablet_copy_client.cc:538] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:23.410897  9860 raft_consensus.cc:1217] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:23.440889 10062 tablet_bootstrap.cc:492] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:23.455503 10060 tablet_bootstrap.cc:492] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:23.456292 10060 tablet_bootstrap.cc:492] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:23.456861 10060 ts_tablet_manager.cc:1403] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.068s	user 0.027s	sys 0.027s
I20260709 13:03:23.459606 10060 raft_consensus.cc:359] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:23.460604 10060 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:23.461388 10060 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:23.464494 10060 ts_tablet_manager.cc:1434] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:03:23.466449  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-c0f89ac8f7b142239d6103bd6ed96037 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:23.466856  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-c0f89ac8f7b142239d6103bd6ed96037 on tablet c0f89ac8f7b142239d6103bd6ed96037 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:23.474179 10062 tablet_bootstrap.cc:492] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:23.475222 10062 tablet_bootstrap.cc:492] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:23.476977 10062 ts_tablet_manager.cc:1403] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.007s
I20260709 13:03:23.480757 10062 raft_consensus.cc:359] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:23.482534 10062 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:23.483124 10062 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:23.484944 10062 ts_tablet_manager.cc:1434] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:23.487089  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-a35f9598c50046539a7f41287876135d received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:23.487552  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-a35f9598c50046539a7f41287876135d on tablet a35f9598c50046539a7f41287876135d with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:23.630633  9860 raft_consensus.cc:1217] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:23.670495  9860 raft_consensus.cc:1217] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:23.736078  9860 raft_consensus.cc:1217] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:23.818684  9935 raft_consensus.cc:1064] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:23.820856  9935 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:23.829905  9860 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:23.831627 10048 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:23.837519  9191 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:23.841486 10048 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:23.844784  9271 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:23.847685  9935 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:23.864118 10048 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:23.878062  9191 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:23.885730  9860 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:23.876627  9271 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:23.900303  9017 catalog_manager.cc:5697] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.948717  9117 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:23.958885  9860 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:23.961218 10048 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:23.962131  9191 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:23.967172 10048 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:23.975786 10033 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:23.989566  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 43e874ff0724409b88c5ec377b73a06a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:23.995929  9860 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:23.999342  9020 catalog_manager.cc:5697] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:23.999207  9198 raft_consensus.cc:2955] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.020998  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e874ff0724409b88c5ec377b73a06a with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:24.028888 10078 tablet_replica.cc:333] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:24.030002 10078 raft_consensus.cc:2243] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:24.032065 10078 raft_consensus.cc:2272] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:24.036005 10078 ts_tablet_manager.cc:1916] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:24.063265 10078 ts_tablet_manager.cc:1929] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:24.064050 10078 log.cc:1199] T 43e874ff0724409b88c5ec377b73a06a P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/43e874ff0724409b88c5ec377b73a06a
I20260709 13:03:24.065975  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 43e874ff0724409b88c5ec377b73a06a (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:24.078227  9903 raft_consensus.cc:1064] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:24.080469  9903 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:24.095446  9198 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:24.098426 10034 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:24.099398  9860 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:24.104136 10034 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:24.108603  9117 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:24.110440 10081 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:24.126933 10081 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.130021  9859 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.140347  9117 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.145882 10080 raft_consensus.cc:1064] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:24.149767  9198 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.160909  9017 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.178311 10080 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:24.182874  9271 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:24.189847  9859 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:24.190107  9198 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:24.197176  9903 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:24.199574  9903 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:24.207803  9122 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:24.210031 10080 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:24.211544  9860 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:24.215866 10080 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:24.219353  9193 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:24.227391 10081 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:24.233527 10081 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.237684  9935 raft_consensus.cc:1064] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:24.252745  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet edfac5554fb144c3ab8a6248c7ea68d8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:24.249656  9859 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.255091  9935 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:24.270057  9020 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.276315 10034 raft_consensus.cc:2955] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.289526  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet edfac5554fb144c3ab8a6248c7ea68d8 with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:24.292395  9860 raft_consensus.cc:2955] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.301564  9198 raft_consensus.cc:2955] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.315766  9115 raft_consensus.cc:2955] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.320127  9018 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.332093 10089 tablet_replica.cc:333] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:24.339810  9271 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:24.362843 10089 raft_consensus.cc:2243] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:24.363778 10089 raft_consensus.cc:2272] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:24.362923  9860 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:24.373626 10080 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:24.379112  9117 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:24.382688 10089 ts_tablet_manager.cc:1916] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:24.387853  9903 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:24.387450 10042 raft_consensus.cc:2955] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.389880  9859 raft_consensus.cc:2955] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.402084  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c0f89ac8f7b142239d6103bd6ed96037 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:24.408416  9860 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:24.409969  9935 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:24.411594  9117 raft_consensus.cc:2955] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.414940  9020 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.419188  9271 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:24.422441 10089 ts_tablet_manager.cc:1929] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:24.427006 10089 log.cc:1199] T edfac5554fb144c3ab8a6248c7ea68d8 P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/edfac5554fb144c3ab8a6248c7ea68d8
I20260709 13:03:24.423556  9198 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:24.428335 10069 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:24.436513  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet edfac5554fb144c3ab8a6248c7ea68d8 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:24.459883 10069 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:24.468467 10048 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.474870  9270 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.487715  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet c0f89ac8f7b142239d6103bd6ed96037 with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:24.500942  9018 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.502262  9860 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.507052 10095 tablet_replica.cc:333] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:24.524874  9198 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.527359 10095 raft_consensus.cc:2243] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:24.528872 10095 pending_rounds.cc:70] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: Trying to abort 1 pending ops.
I20260709 13:03:24.529671 10095 pending_rounds.cc:77] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305634628224229376 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } } }
I20260709 13:03:24.531051 10095 raft_consensus.cc:2917] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:03:24.531773 10095 raft_consensus.cc:2889] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:03:24.532140 10095 raft_consensus.cc:2272] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:24.545643 10095 ts_tablet_manager.cc:1916] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:24.549888  9117 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:24.574664  9194 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:24.575626 10095 ts_tablet_manager.cc:1929] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:24.576166 10095 log.cc:1199] T c0f89ac8f7b142239d6103bd6ed96037 P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/c0f89ac8f7b142239d6103bd6ed96037
I20260709 13:03:24.580276  9860 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:24.581090 10048 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:24.583791 10048 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:24.587283  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet c0f89ac8f7b142239d6103bd6ed96037 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:24.591584  9935 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.598646  9860 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.600298  9193 raft_consensus.cc:2955] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:24.607702  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a35f9598c50046539a7f41287876135d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:24.622982  9018 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:24.646183  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet a35f9598c50046539a7f41287876135d with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:24.653723 10096 tablet_replica.cc:333] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:24.654896 10096 raft_consensus.cc:2243] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:24.655905 10096 raft_consensus.cc:2272] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:24.659924 10096 ts_tablet_manager.cc:1916] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:24.674623 10096 ts_tablet_manager.cc:1929] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:24.675066 10096 log.cc:1199] T a35f9598c50046539a7f41287876135d P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/a35f9598c50046539a7f41287876135d
I20260709 13:03:24.676833  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet a35f9598c50046539a7f41287876135d (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:25.732544  9194 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:25.740594  9122 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:25.742098  9270 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:25.742925  9164 consensus_peers.cc:597] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa5405d9d7b34438b7f5732807fac54a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:25.744072 10092 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.746635 10100 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:25.756975 10101 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.759059  9271 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.761265  9115 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.779250  9020 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:25.779716  9271 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:25.792009  9115 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:25.792470  9194 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:25.793239  9238 consensus_peers.cc:597] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d9ec1fba08a49aeb3e1a8a52e798291. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:25.794147  9903 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:25.795369 10098 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.816069  9903 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.820276  9115 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.830174  9122 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:25.838449  9193 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.841281  9021 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:03:25.848111  9089 consensus_peers.cc:597] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 909051b50b2c464ea2b13a15a243817c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:25.854377  9193 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:25.861888  9935 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.857690  9271 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:25.865995  9935 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.870833 10097 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.874411  9194 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.883693  9271 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.886232  9270 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:25.890096  9018 catalog_manager.cc:5697] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:25.906036  9122 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:25.908231 10034 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:25.914492  9238 consensus_peers.cc:597] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1fdaa422821a46b8bc73801d1abfb3c0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:25.916247  9198 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:25.923727  9903 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:25.926002  9903 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.927982  9115 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.955955  9194 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:25.966912  9020 catalog_manager.cc:5697] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:26.273938 10119 ts_tablet_manager.cc:933] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:26.279249 10119 tablet_copy_client.cc:323] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:26.281117  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 8d9ec1fba08a49aeb3e1a8a52e798291 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:26.281737  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 8d9ec1fba08a49aeb3e1a8a52e798291 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-8d9ec1fba08a49aeb3e1a8a52e798291
I20260709 13:03:26.308326  9282 tablet_copy_source_session.cc:215] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:26.311439 10121 ts_tablet_manager.cc:933] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:26.318583 10119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d9ec1fba08a49aeb3e1a8a52e798291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.321703 10121 tablet_copy_client.cc:323] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:26.324616  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet 909051b50b2c464ea2b13a15a243817c from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:26.325209  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet 909051b50b2c464ea2b13a15a243817c from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-909051b50b2c464ea2b13a15a243817c
I20260709 13:03:26.365643  9133 tablet_copy_source_session.cc:215] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:26.367887 10119 tablet_copy_client.cc:806] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:26.368571 10119 tablet_copy_client.cc:670] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:26.372584 10122 ts_tablet_manager.cc:933] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:26.373308 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909051b50b2c464ea2b13a15a243817c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.392760 10122 tablet_copy_client.cc:323] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:26.395454  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 1fdaa422821a46b8bc73801d1abfb3c0 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:26.396132  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 1fdaa422821a46b8bc73801d1abfb3c0 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-1fdaa422821a46b8bc73801d1abfb3c0
I20260709 13:03:26.397526 10124 ts_tablet_manager.cc:933] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:26.395491 10119 tablet_copy_client.cc:538] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:26.414096  9282 tablet_copy_source_session.cc:215] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:26.417184 10121 tablet_copy_client.cc:806] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:26.417814 10121 tablet_copy_client.cc:670] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:26.419646 10124 tablet_copy_client.cc:323] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:26.421451  9208 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet fa5405d9d7b34438b7f5732807fac54a from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:26.422051  9208 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet fa5405d9d7b34438b7f5732807fac54a from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-fa5405d9d7b34438b7f5732807fac54a
I20260709 13:03:26.426834 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fdaa422821a46b8bc73801d1abfb3c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.427920 10119 tablet_bootstrap.cc:492] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:26.430202 10121 tablet_copy_client.cc:538] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:26.458105 10121 tablet_bootstrap.cc:492] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:26.459535  9208 tablet_copy_source_session.cc:215] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:26.473091 10124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5405d9d7b34438b7f5732807fac54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:26.492440 10122 tablet_copy_client.cc:806] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:26.501581 10122 tablet_copy_client.cc:670] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:26.500903 10119 tablet_bootstrap.cc:492] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:26.503999 10119 tablet_bootstrap.cc:492] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:26.505870 10119 ts_tablet_manager.cc:1403] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.078s	user 0.024s	sys 0.035s
I20260709 13:03:26.508209 10122 tablet_copy_client.cc:538] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:26.509804 10119 raft_consensus.cc:359] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:26.510885 10119 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:26.511714 10119 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:26.514394 10124 tablet_copy_client.cc:806] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:26.514979 10124 tablet_copy_client.cc:670] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:26.524504 10124 tablet_copy_client.cc:538] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:26.541603 10124 tablet_bootstrap.cc:492] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:26.542160 10122 tablet_bootstrap.cc:492] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:26.545472 10119 ts_tablet_manager.cc:1434] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:03:26.547861  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-8d9ec1fba08a49aeb3e1a8a52e798291 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:26.548449  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-8d9ec1fba08a49aeb3e1a8a52e798291 on tablet 8d9ec1fba08a49aeb3e1a8a52e798291 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:26.572018 10124 tablet_bootstrap.cc:492] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:26.573200 10124 tablet_bootstrap.cc:492] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:26.574059 10124 ts_tablet_manager.cc:1403] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.033s	user 0.025s	sys 0.004s
I20260709 13:03:26.576479 10121 tablet_bootstrap.cc:492] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:26.577412 10121 tablet_bootstrap.cc:492] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:26.578084 10121 ts_tablet_manager.cc:1403] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.120s	user 0.029s	sys 0.000s
I20260709 13:03:26.577523 10124 raft_consensus.cc:359] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:26.578751 10124 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:26.579560 10124 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:26.582581 10124 ts_tablet_manager.cc:1434] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:03:26.582983 10122 tablet_bootstrap.cc:492] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:26.583986 10122 tablet_bootstrap.cc:492] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:26.584689 10122 ts_tablet_manager.cc:1403] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.017s
I20260709 13:03:26.586521  9208 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-fa5405d9d7b34438b7f5732807fac54a received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:26.586141 10121 raft_consensus.cc:359] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:26.586964  9208 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-fa5405d9d7b34438b7f5732807fac54a on tablet fa5405d9d7b34438b7f5732807fac54a with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:26.587528 10121 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:26.588490 10121 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:26.589314 10122 raft_consensus.cc:359] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:26.590236 10122 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:26.590965 10122 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:26.592455 10121 ts_tablet_manager.cc:1434] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:03:26.593799 10122 ts_tablet_manager.cc:1434] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:03:26.594288  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-909051b50b2c464ea2b13a15a243817c received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:26.594760  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-909051b50b2c464ea2b13a15a243817c on tablet 909051b50b2c464ea2b13a15a243817c with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:26.595992  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-1fdaa422821a46b8bc73801d1abfb3c0 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:26.596475  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-1fdaa422821a46b8bc73801d1abfb3c0 on tablet 1fdaa422821a46b8bc73801d1abfb3c0 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:26.691839  9860 raft_consensus.cc:1217] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:26.749001  9859 raft_consensus.cc:1217] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:26.836699  9859 raft_consensus.cc:1217] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:26.903506  9859 raft_consensus.cc:1217] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:27.207329 10132 raft_consensus.cc:1064] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:27.210682 10132 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:27.221100 10135 raft_consensus.cc:1064] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:27.225580  9122 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.226850  9860 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.230060 10135 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:27.231441 10136 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:27.232589  9194 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.233804 10134 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:27.246989 10134 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:27.297394  9115 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.300135 10141 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:03:27.312713  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 66999.9us
I20260709 13:03:27.315443  9859 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.324663  9194 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.330891 10141 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:27.337960 10034 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:27.355635  9903 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.368285 10133 raft_consensus.cc:1064] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:27.371487 10133 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:27.375324  9115 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.387145 10134 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.389761  9859 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.395217  9860 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.396600  9194 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.400084  9020 catalog_manager.cc:5697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:27.414981  9018 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.422992  9191 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.426443  9122 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.439646  9267 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.441725  9859 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.443686 10100 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:27.446545 10065 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:27.459101 10097 raft_consensus.cc:1064] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:27.461071  9115 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.462716 10133 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:03:27.462052 10097 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:27.470928 10148 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.487094  9860 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.493654  9267 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.510524  9859 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.510705  9267 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.512912  9194 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:27.514096 10097 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:27.515363 10069 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:27.518658  9935 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:27.536255  9115 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.549221 10155 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.550438  9860 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.550921  9267 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.564129  9194 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.592410  9018 catalog_manager.cc:5697] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.603355  9018 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.631204  9194 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:27.645834  9115 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:27.654845  9271 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:27.649813  9267 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:27.661007  9860 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:27.662289  9859 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:27.662887 10100 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:27.663381  9191 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:27.664544 10155 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:27.667367  9859 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:27.669401  9935 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:27.669859 10134 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:27.673708  9191 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:27.675302  9115 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:27.676741 10141 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:27.678944 10141 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:27.682005 10133 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.683090  9862 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:27.693264 10034 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.694761  9862 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.700883  9265 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:27.703684  9115 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.716125  9903 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:27.719408 10136 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.731586  9017 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.732439  9859 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:03:27.735358  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52490.8us
I20260709 13:03:27.705250 10148 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:27.705143 10069 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.744311  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5405d9d7b34438b7f5732807fac54a with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:27.749588  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fa5405d9d7b34438b7f5732807fac54a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:27.757277  9191 raft_consensus.cc:2955] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.761626  9270 raft_consensus.cc:2955] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:03:27.771906  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 53470.9us
I20260709 13:03:27.776798  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 909051b50b2c464ea2b13a15a243817c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:27.778872  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8d9ec1fba08a49aeb3e1a8a52e798291 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:27.783653  9017 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:27.785874  9860 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.786166  9021 catalog_manager.cc:5697] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:27.807971  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet 8d9ec1fba08a49aeb3e1a8a52e798291 with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:27.809237  9862 raft_consensus.cc:2955] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.850498 10165 tablet_replica.cc:333] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:27.857903  9017 catalog_manager.cc:5697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.871336  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1fdaa422821a46b8bc73801d1abfb3c0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:27.872190  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 909051b50b2c464ea2b13a15a243817c with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:27.876946  9193 raft_consensus.cc:2955] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:27.878288  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fdaa422821a46b8bc73801d1abfb3c0 with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:27.881064 10164 tablet_replica.cc:333] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:27.888592 10165 raft_consensus.cc:2243] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:27.915450 10165 raft_consensus.cc:2272] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:27.901049 10164 raft_consensus.cc:2243] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:27.923157 10164 raft_consensus.cc:2272] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:27.938988 10164 ts_tablet_manager.cc:1916] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:27.938988 10165 ts_tablet_manager.cc:1916] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:27.952718 10166 tablet_replica.cc:333] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:27.953894 10166 raft_consensus.cc:2243] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:27.954757 10166 raft_consensus.cc:2272] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:27.958855 10166 ts_tablet_manager.cc:1916] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:27.964838 10165 ts_tablet_manager.cc:1929] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:27.965329 10165 log.cc:1199] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/8d9ec1fba08a49aeb3e1a8a52e798291
I20260709 13:03:27.966763  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet 8d9ec1fba08a49aeb3e1a8a52e798291 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:27.969419 10164 ts_tablet_manager.cc:1929] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:27.969821 10164 log.cc:1199] T fa5405d9d7b34438b7f5732807fac54a P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/fa5405d9d7b34438b7f5732807fac54a
I20260709 13:03:27.971212  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet fa5405d9d7b34438b7f5732807fac54a (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:27.971333 10164 tablet_replica.cc:333] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:27.972465 10164 raft_consensus.cc:2243] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:27.973057 10164 raft_consensus.cc:2272] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:27.973553 10166 ts_tablet_manager.cc:1929] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:27.974005 10166 log.cc:1199] T 909051b50b2c464ea2b13a15a243817c P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/909051b50b2c464ea2b13a15a243817c
I20260709 13:03:27.975677  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 909051b50b2c464ea2b13a15a243817c (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:27.976064 10164 ts_tablet_manager.cc:1916] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:27.988226 10164 ts_tablet_manager.cc:1929] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:27.988531 10164 log.cc:1199] T 1fdaa422821a46b8bc73801d1abfb3c0 P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/1fdaa422821a46b8bc73801d1abfb3c0
I20260709 13:03:27.990101  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet 1fdaa422821a46b8bc73801d1abfb3c0 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:29.035522  9271 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.044643  9117 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:29.044711  9194 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:29.045101  9238 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9daa912f9e08438caf063c0c83e09fe5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:29.046823 10134 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:29.047989 10169 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:29.058589 10134 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.061371  9194 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.068500  9117 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.075666  9115 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.086969  9017 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:29.099730  9089 consensus_peers.cc:597] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23636a3a86df4bdf91dc6a7ffe723527. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:29.103600  9271 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:29.108081 10069 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:29.109684  9194 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:29.111241  9935 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:29.132992 10069 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.144294  9194 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.145150  9271 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.158308  9020 catalog_manager.cc:5697] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:29.162376  9267 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.185899  9117 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:29.187383 10169 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:29.188854  9194 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:29.189193  9238 consensus_peers.cc:597] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a08146adfeb9417ca7067de38da555b0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:29.190357 10169 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:29.195711 10141 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.197415  9117 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.223083  9198 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.223800  9194 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.230374  9020 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:03:29.285341  9164 consensus_peers.cc:597] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a06199955d24d13ba9589faa3f11ebe. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:29.288658  9122 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:29.293193 10168 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:29.303079 10163 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.305065  9122 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:03:29.320320  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 82930.6us
I20260709 13:03:29.331578  9017 catalog_manager.cc:5697] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.332494  9270 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:29.336444 10187 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:29.348563  9270 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:29.462224 10191 ts_tablet_manager.cc:933] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:29.465014 10191 tablet_copy_client.cc:323] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:29.469046  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 9daa912f9e08438caf063c0c83e09fe5 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:29.469732  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 9daa912f9e08438caf063c0c83e09fe5 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-9daa912f9e08438caf063c0c83e09fe5
I20260709 13:03:29.478668  9282 tablet_copy_source_session.cc:215] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:29.481755 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9daa912f9e08438caf063c0c83e09fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:29.496371 10191 tablet_copy_client.cc:806] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:29.497143 10191 tablet_copy_client.cc:670] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:29.504719 10191 tablet_copy_client.cc:538] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:29.516141 10191 tablet_bootstrap.cc:492] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:29.552531 10191 tablet_bootstrap.cc:492] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:29.553646 10191 tablet_bootstrap.cc:492] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:29.554458 10191 ts_tablet_manager.cc:1403] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.006s
I20260709 13:03:29.557636 10191 raft_consensus.cc:359] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.558377 10191 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:29.559008 10191 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.562987 10191 ts_tablet_manager.cc:1434] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:03:29.565635  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-9daa912f9e08438caf063c0c83e09fe5 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:29.566171  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-9daa912f9e08438caf063c0c83e09fe5 on tablet 9daa912f9e08438caf063c0c83e09fe5 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:29.617899 10191 ts_tablet_manager.cc:933] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:29.620401 10191 tablet_copy_client.cc:323] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:29.622006  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet a08146adfeb9417ca7067de38da555b0 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:29.622531  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet a08146adfeb9417ca7067de38da555b0 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-a08146adfeb9417ca7067de38da555b0
I20260709 13:03:29.634797  9282 tablet_copy_source_session.cc:215] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:29.638494 10191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a08146adfeb9417ca7067de38da555b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:29.653476 10196 ts_tablet_manager.cc:933] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:29.655958 10196 tablet_copy_client.cc:323] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:29.658147  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet 23636a3a86df4bdf91dc6a7ffe723527 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:29.658784  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet 23636a3a86df4bdf91dc6a7ffe723527 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-23636a3a86df4bdf91dc6a7ffe723527
I20260709 13:03:29.670660  9133 tablet_copy_source_session.cc:215] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:29.672959 10191 tablet_copy_client.cc:806] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:29.673713 10191 tablet_copy_client.cc:670] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:29.679070 10191 tablet_copy_client.cc:538] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:29.681499 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23636a3a86df4bdf91dc6a7ffe723527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:29.690054 10191 tablet_bootstrap.cc:492] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:29.708894 10196 tablet_copy_client.cc:806] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:29.709649 10196 tablet_copy_client.cc:670] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:29.725404 10191 tablet_bootstrap.cc:492] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:29.726601 10191 tablet_bootstrap.cc:492] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:29.727458 10191 ts_tablet_manager.cc:1403] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.009s
I20260709 13:03:29.733563 10191 raft_consensus.cc:359] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.734619 10191 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:29.735436 10191 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.739823 10199 ts_tablet_manager.cc:933] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:29.740136 10196 tablet_copy_client.cc:538] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:29.742925 10191 ts_tablet_manager.cc:1434] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:03:29.745683  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-a08146adfeb9417ca7067de38da555b0 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:29.746111  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-a08146adfeb9417ca7067de38da555b0 on tablet a08146adfeb9417ca7067de38da555b0 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:29.756285 10196 tablet_bootstrap.cc:492] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:29.760619 10199 tablet_copy_client.cc:323] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:29.764317  9208 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet 0a06199955d24d13ba9589faa3f11ebe from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:29.764899  9208 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet 0a06199955d24d13ba9589faa3f11ebe from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-0a06199955d24d13ba9589faa3f11ebe
I20260709 13:03:29.775585  9208 tablet_copy_source_session.cc:215] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:29.779589 10199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a06199955d24d13ba9589faa3f11ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:29.793922 10196 tablet_bootstrap.cc:492] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:29.797411 10196 tablet_bootstrap.cc:492] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:29.798210 10196 ts_tablet_manager.cc:1403] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.042s	user 0.031s	sys 0.000s
I20260709 13:03:29.806233 10199 tablet_copy_client.cc:806] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:29.807047 10199 tablet_copy_client.cc:670] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:29.806238 10196 raft_consensus.cc:359] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.808000 10196 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:29.808786 10196 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.811522 10199 tablet_copy_client.cc:538] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:29.811627 10196 ts_tablet_manager.cc:1434] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:03:29.813786  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-23636a3a86df4bdf91dc6a7ffe723527 received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:29.814268  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-23636a3a86df4bdf91dc6a7ffe723527 on tablet 23636a3a86df4bdf91dc6a7ffe723527 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:29.823355 10199 tablet_bootstrap.cc:492] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:29.846661 10199 tablet_bootstrap.cc:492] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:29.847605 10199 tablet_bootstrap.cc:492] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:29.848277 10199 ts_tablet_manager.cc:1403] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.009s
I20260709 13:03:29.850905 10199 raft_consensus.cc:359] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.851600 10199 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:29.852300 10199 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:29.854745 10199 ts_tablet_manager.cc:1434] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:03:29.858249  9208 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-0a06199955d24d13ba9589faa3f11ebe received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:29.858701  9208 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-0a06199955d24d13ba9589faa3f11ebe on tablet 0a06199955d24d13ba9589faa3f11ebe with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:29.942449  9862 raft_consensus.cc:1217] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:30.037295  9862 raft_consensus.cc:1217] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:30.153092  9862 raft_consensus.cc:1217] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:30.242554  9862 raft_consensus.cc:1217] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:30.438827 10134 raft_consensus.cc:1064] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:30.441201 10134 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:30.447731  9862 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.448321  9194 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.449483  9122 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.449218 10134 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.450286 10185 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.451079 10169 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.463217 10134 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.467446  9194 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.476841  9122 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.485728  9020 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.494299  9862 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.535697  9270 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:30.548162  9862 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:30.548207  9122 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:30.550370 10185 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:30.551913 10134 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:30.564539 10212 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.566484  9115 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.581619  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9daa912f9e08438caf063c0c83e09fe5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:30.589372  9862 raft_consensus.cc:2955] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.592839  9021 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:30.608597  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet 9daa912f9e08438caf063c0c83e09fe5 with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:30.614437 10180 raft_consensus.cc:1064] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:30.617172 10180 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:30.630352  9862 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.630934  9267 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.631948 10069 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.632411  9194 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.634192 10069 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.630048 10213 tablet_replica.cc:333] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:30.637540 10180 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.639462 10213 raft_consensus.cc:2243] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:30.657205 10213 raft_consensus.cc:2272] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.656164  9935 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.666462 10163 raft_consensus.cc:1064] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:30.669854  9194 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.674115 10213 ts_tablet_manager.cc:1916] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:30.697530 10213 ts_tablet_manager.cc:1929] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:30.698133 10213 log.cc:1199] T 9daa912f9e08438caf063c0c83e09fe5 P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/9daa912f9e08438caf063c0c83e09fe5
I20260709 13:03:30.702368  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet 9daa912f9e08438caf063c0c83e09fe5 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:30.706933  9267 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.716060  9018 catalog_manager.cc:5697] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.717700 10163 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:30.730252  9862 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.731783 10212 raft_consensus.cc:1064] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:30.736874 10212 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:30.759114  9862 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.760563 10168 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.764271  9194 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.764896  9115 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.766405  9859 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.766346 10212 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.768092  9115 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.767931 10169 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.769696 10168 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:30.772166  9267 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:30.774925 10212 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:30.775499 10188 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:30.798867  9122 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:30.813081  9862 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:30.814527  9191 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:30.816397 10219 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:30.818912 10218 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:30.840075 10180 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.849727  9191 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.863974  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 23636a3a86df4bdf91dc6a7ffe723527 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:30.865249  9862 raft_consensus.cc:2955] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.878990  9020 catalog_manager.cc:5697] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:30.888154 10169 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.897622  9859 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.901134  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 23636a3a86df4bdf91dc6a7ffe723527 with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:30.906034  9860 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.907585  9194 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.903600 10163 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.928624  9017 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:30.942811  9020 catalog_manager.cc:5697] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:30.955065  9265 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:03:31.005188  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 68755.5us
I20260709 13:03:31.013310  9122 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:31.041131 10228 tablet_replica.cc:333] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:31.042457 10228 raft_consensus.cc:2243] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.043473 10228 raft_consensus.cc:2272] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.075192  9198 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:31.087247  9114 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:31.089365 10228 ts_tablet_manager.cc:1916] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:31.092473  9860 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:31.094238 10229 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:31.101930 10163 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:31.108188 10228 ts_tablet_manager.cc:1929] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:31.108933 10228 log.cc:1199] T 23636a3a86df4bdf91dc6a7ffe723527 P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/23636a3a86df4bdf91dc6a7ffe723527
I20260709 13:03:31.108340  9860 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:31.115552  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a06199955d24d13ba9589faa3f11ebe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:31.119868  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 23636a3a86df4bdf91dc6a7ffe723527 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:31.120373  9265 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:31.122912  9017 catalog_manager.cc:5697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:31.123474  9267 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:31.128023 10230 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:31.132864  9270 raft_consensus.cc:2955] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:31.140556  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a06199955d24d13ba9589faa3f11ebe with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:31.142849  9859 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:31.147351 10134 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:31.150770  9117 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:31.155428 10169 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:31.159965 10169 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:31.169596  9859 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:31.172602 10232 tablet_replica.cc:333] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:31.173801  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a08146adfeb9417ca7067de38da555b0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:31.183174 10232 raft_consensus.cc:2243] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.184368 10232 raft_consensus.cc:2272] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.191046 10232 ts_tablet_manager.cc:1916] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:31.193023  9020 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:31.198104  9114 raft_consensus.cc:2955] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:31.208827 10232 ts_tablet_manager.cc:1929] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:31.209411 10232 log.cc:1199] T 0a06199955d24d13ba9589faa3f11ebe P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/0a06199955d24d13ba9589faa3f11ebe
I20260709 13:03:31.211457  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet 0a06199955d24d13ba9589faa3f11ebe (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:31.220782  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet a08146adfeb9417ca7067de38da555b0 with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:31.222877 10233 tablet_replica.cc:333] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:31.225301 10233 raft_consensus.cc:2243] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.228371 10233 raft_consensus.cc:2272] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.234383 10233 ts_tablet_manager.cc:1916] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:31.250422 10233 ts_tablet_manager.cc:1929] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:31.250780 10233 log.cc:1199] T a08146adfeb9417ca7067de38da555b0 P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/a08146adfeb9417ca7067de38da555b0
I20260709 13:03:31.252058  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet a08146adfeb9417ca7067de38da555b0 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:32.289553  9198 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:32.296404  9164 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7597d602dc674623ae1d5de5fdb5d2ee. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:32.299307  9270 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:32.300712  9114 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:32.300801 10236 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:32.302163 10237 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:32.308813 10236 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.310554  9270 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.311525  9114 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.327899  9018 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:32.330744  9198 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:32.339892  9270 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:32.340371  9164 consensus_peers.cc:597] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db3bbfae79ea4f4cb8b7e3079bdb3908. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:32.341508 10237 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:32.342585  9114 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:32.345052 10236 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:32.354753 10237 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.362955  9270 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.379076  9114 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.383635  9017 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.387969  9198 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:32.406157  9164 consensus_peers.cc:597] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e2655d16bd349309dab52831e66fee8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:32.408058  9114 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:32.409442 10237 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:32.410508  9270 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:32.411913 10237 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:03:32.419835 10237 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.424485  9117 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.431341  9270 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.434038  9265 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:32.440482  9018 catalog_manager.cc:5697] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:03:32.448936  9238 consensus_peers.cc:597] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca0b93b18a7f4e6398018531838527d9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:32.453521  9198 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:32.454768  9117 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:32.456239 10185 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:32.459649 10169 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:32.470352 10185 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.471827  9117 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.474920  9198 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.489750  9018 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:32.834848 10255 ts_tablet_manager.cc:933] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:32.837039 10255 tablet_copy_client.cc:323] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:32.838646  9208 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet 7597d602dc674623ae1d5de5fdb5d2ee from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:32.839102  9208 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet 7597d602dc674623ae1d5de5fdb5d2ee from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-7597d602dc674623ae1d5de5fdb5d2ee
I20260709 13:03:32.844396  9208 tablet_copy_source_session.cc:215] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:32.847044 10255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7597d602dc674623ae1d5de5fdb5d2ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:32.863607 10255 tablet_copy_client.cc:806] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:32.864240 10255 tablet_copy_client.cc:670] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:32.868541 10255 tablet_copy_client.cc:538] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:32.877153 10255 tablet_bootstrap.cc:492] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:32.898826 10255 tablet_bootstrap.cc:492] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:32.899479 10255 tablet_bootstrap.cc:492] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:32.900041 10255 ts_tablet_manager.cc:1403] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.023s	user 0.021s	sys 0.000s
I20260709 13:03:32.902494 10255 raft_consensus.cc:359] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:32.903182 10255 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:32.903932 10255 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:32.906733 10255 ts_tablet_manager.cc:1434] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:03:32.909030  9208 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-7597d602dc674623ae1d5de5fdb5d2ee received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:32.909452  9208 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-7597d602dc674623ae1d5de5fdb5d2ee on tablet 7597d602dc674623ae1d5de5fdb5d2ee with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:32.917045 10255 ts_tablet_manager.cc:933] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:32.921603 10255 tablet_copy_client.cc:323] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:32.923049  9208 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet 9e2655d16bd349309dab52831e66fee8 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:32.923574  9208 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet 9e2655d16bd349309dab52831e66fee8 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-9e2655d16bd349309dab52831e66fee8
I20260709 13:03:32.939533  9208 tablet_copy_source_session.cc:215] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:32.943209 10255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2655d16bd349309dab52831e66fee8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:32.960804 10261 ts_tablet_manager.cc:933] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:32.961585 10260 ts_tablet_manager.cc:933] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:32.973783 10261 tablet_copy_client.cc:323] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:32.976294  9208 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet db3bbfae79ea4f4cb8b7e3079bdb3908 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:32.976843  9208 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet db3bbfae79ea4f4cb8b7e3079bdb3908 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-db3bbfae79ea4f4cb8b7e3079bdb3908
I20260709 13:03:32.984350 10255 tablet_copy_client.cc:806] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:32.984637  9208 tablet_copy_source_session.cc:215] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:32.985149 10255 tablet_copy_client.cc:670] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:32.988005 10260 tablet_copy_client.cc:323] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:32.988768 10261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3bbfae79ea4f4cb8b7e3079bdb3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:32.989080 10255 tablet_copy_client.cc:538] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:32.989815  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet ca0b93b18a7f4e6398018531838527d9 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:32.990355  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet ca0b93b18a7f4e6398018531838527d9 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-ca0b93b18a7f4e6398018531838527d9
I20260709 13:03:32.999864 10255 tablet_bootstrap.cc:492] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:33.006659  9282 tablet_copy_source_session.cc:215] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:33.010872 10260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca0b93b18a7f4e6398018531838527d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:33.012959 10261 tablet_copy_client.cc:806] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:33.013616 10261 tablet_copy_client.cc:670] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:33.021910 10261 tablet_copy_client.cc:538] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:33.034224 10261 tablet_bootstrap.cc:492] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:33.081214 10255 tablet_bootstrap.cc:492] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:33.099514 10255 tablet_bootstrap.cc:492] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:33.100325 10255 ts_tablet_manager.cc:1403] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.101s	user 0.035s	sys 0.013s
I20260709 13:03:33.136924 10255 raft_consensus.cc:359] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:33.147915 10255 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:33.151212 10255 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:33.154685 10255 ts_tablet_manager.cc:1434] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 13:03:33.168303 10260 tablet_copy_client.cc:806] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:33.172839  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-9e2655d16bd349309dab52831e66fee8 received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:33.173326  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-9e2655d16bd349309dab52831e66fee8 on tablet 9e2655d16bd349309dab52831e66fee8 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:33.171075 10260 tablet_copy_client.cc:670] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
W20260709 13:03:33.176887  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 69647.9us
I20260709 13:03:33.183598 10260 tablet_copy_client.cc:538] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:33.195835 10260 tablet_bootstrap.cc:492] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:33.203896 10261 tablet_bootstrap.cc:492] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:33.205024 10261 tablet_bootstrap.cc:492] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:33.205747 10261 ts_tablet_manager.cc:1403] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.172s	user 0.054s	sys 0.039s
I20260709 13:03:33.210105 10261 raft_consensus.cc:359] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:33.211002 10261 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:33.211688 10261 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:33.217886 10261 ts_tablet_manager.cc:1434] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:03:33.219589  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-db3bbfae79ea4f4cb8b7e3079bdb3908 received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:33.220010  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-db3bbfae79ea4f4cb8b7e3079bdb3908 on tablet db3bbfae79ea4f4cb8b7e3079bdb3908 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:33.225919 10260 tablet_bootstrap.cc:492] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:33.227239 10260 tablet_bootstrap.cc:492] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:33.228091 10260 ts_tablet_manager.cc:1403] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.012s
I20260709 13:03:33.231447 10260 raft_consensus.cc:359] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:33.232447 10260 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:33.233181 10260 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:33.237128 10260 ts_tablet_manager.cc:1434] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:03:33.239940  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-ca0b93b18a7f4e6398018531838527d9 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:33.240391  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-ca0b93b18a7f4e6398018531838527d9 on tablet ca0b93b18a7f4e6398018531838527d9 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:33.362481  9861 raft_consensus.cc:1217] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:33.384827  9861 raft_consensus.cc:1217] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:33.396829  9861 raft_consensus.cc:1217] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:33.444932  9861 raft_consensus.cc:1217] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:03:33.769368 10254 raft_consensus.cc:1064] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:33.771926 10254 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:33.783300  9114 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:33.783919  9861 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:33.785125 10169 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:33.789058  9198 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:33.790016 10271 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:33.792716 10168 raft_consensus.cc:1064] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:33.794071 10271 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:33.795101 10168 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:33.823158  9860 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:33.824661  9270 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:33.827147  9114 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:33.823977 10254 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:33.827716  9198 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:33.838630 10168 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:33.840443 10163 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:33.843854 10237 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:33.856345  9020 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:33.860713  9114 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:33.861428  9860 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:33.886162 10163 raft_consensus.cc:1064] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:33.900501 10168 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:33.905156  9270 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:33.907050 10163 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:33.945662  9122 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:33.947602  9861 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:33.974342  9860 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:33.975878 10236 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:33.985412  9122 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:33.987205 10168 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:33.988828  9270 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:33.991446 10236 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:34.002667 10163 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.004556  9122 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.017143 10236 raft_consensus.cc:1064] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:34.022925 10236 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:34.049329  9270 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.057518  9860 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.079593  9857 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:34.081768 10168 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:34.082073  9270 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:34.082445  9122 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:34.083621 10163 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:34.087997 10168 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:34.107209  9017 catalog_manager.cc:5697] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.109555  9017 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.124037 10236 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.128862  9122 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.148833  9191 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:34.149633  9194 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:34.154992  9265 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:34.165223  9861 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:34.168437  9115 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:34.171929 10276 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:34.171967 10237 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:03:34.175086  9860 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:34.176831 10168 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:34.187436 10169 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.194226 10168 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.204511  9861 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:34.206202  9862 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.211408  9265 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:34.212049  9270 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.212523  9115 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.228672 10168 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:03:34.234300 10276 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:03:34.243587 10237 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.247583  9021 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:34.249478  9858 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.231974  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ca0b93b18a7f4e6398018531838527d9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:34.250290  9861 raft_consensus.cc:2955] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.263001  9018 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:34.281203  9021 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:34.293794 10292 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:03:34.296962  9267 raft_consensus.cc:2955] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:03:34.297909  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 129644us
I20260709 13:03:34.301460  9122 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:34.305967  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7597d602dc674623ae1d5de5fdb5d2ee with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:34.313395  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet ca0b93b18a7f4e6398018531838527d9 with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:34.317680 10168 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:34.320847  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9e2655d16bd349309dab52831e66fee8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:34.320384  9862 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.333149  9122 raft_consensus.cc:2955] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.383260 10296 tablet_replica.cc:333] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:34.384591 10296 raft_consensus.cc:2243] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:34.385658 10296 raft_consensus.cc:2272] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:34.405658 10296 ts_tablet_manager.cc:1916] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:34.416508  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 7597d602dc674623ae1d5de5fdb5d2ee with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:34.416757  9021 catalog_manager.cc:5697] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.426396 10298 tablet_replica.cc:333] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:34.436825 10298 raft_consensus.cc:2243] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:34.437768 10298 raft_consensus.cc:2272] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:34.438549 10296 ts_tablet_manager.cc:1929] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:34.439690 10296 log.cc:1199] T ca0b93b18a7f4e6398018531838527d9 P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/ca0b93b18a7f4e6398018531838527d9
I20260709 13:03:34.442061  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet ca0b93b18a7f4e6398018531838527d9 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:34.443545 10298 ts_tablet_manager.cc:1916] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:34.463075  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet 9e2655d16bd349309dab52831e66fee8 with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:34.466180 10299 tablet_replica.cc:333] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:34.467480 10299 raft_consensus.cc:2243] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:34.468355 10299 raft_consensus.cc:2272] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:34.474138  9194 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:34.480402 10299 ts_tablet_manager.cc:1916] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:34.487402 10298 ts_tablet_manager.cc:1929] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:34.488015 10298 log.cc:1199] T 7597d602dc674623ae1d5de5fdb5d2ee P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/7597d602dc674623ae1d5de5fdb5d2ee
I20260709 13:03:34.489944  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 7597d602dc674623ae1d5de5fdb5d2ee (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:34.493433  9862 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:34.493460  9122 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:34.500872 10295 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:34.502873 10295 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:34.509481 10299 ts_tablet_manager.cc:1929] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:34.510018 10299 log.cc:1199] T 9e2655d16bd349309dab52831e66fee8 P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/9e2655d16bd349309dab52831e66fee8
I20260709 13:03:34.509056 10294 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.511967  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet 9e2655d16bd349309dab52831e66fee8 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:34.522929  9858 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.531945  9115 raft_consensus.cc:2955] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:34.539047  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet db3bbfae79ea4f4cb8b7e3079bdb3908 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:34.546912  9021 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:34.564122  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet db3bbfae79ea4f4cb8b7e3079bdb3908 with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:34.565306 10298 tablet_replica.cc:333] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:34.566424 10298 raft_consensus.cc:2243] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:34.567445 10298 raft_consensus.cc:2272] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:34.574445 10298 ts_tablet_manager.cc:1916] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:34.587826 10298 ts_tablet_manager.cc:1929] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:03:34.588217 10298 log.cc:1199] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/db3bbfae79ea4f4cb8b7e3079bdb3908
I20260709 13:03:34.589690  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet db3bbfae79ea4f4cb8b7e3079bdb3908 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:35.638844  9191 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:35.649972  9270 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:35.649983  9164 consensus_peers.cc:597] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 06c3356087cc45e78fa0f79168b68e8c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:35.651538 10294 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:35.652864  9122 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:35.654275 10294 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:35.662452 10294 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.664743  9270 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.666534  9122 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.682726  9021 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:35.678077  9191 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:35.694772  9267 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:35.696264 10294 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:35.697402  9164 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0917335c1234d5abec1e40c742de6be. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:35.702265  9122 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:35.704756 10304 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:35.713244 10304 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.715077  9267 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.720201  9122 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.735065  9115 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:35.747298  9018 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:03:35.750072  9089 consensus_peers.cc:597] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 709565930ea044f08322d94e29d83adf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:35.753640  9267 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:35.754480  9191 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:35.755919 10219 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:35.757932 10297 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:35.769948 10219 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.773738  9191 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.775386  9267 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.784855  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:35.783969  9270 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:35.791203  9021 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:35.807304  9190 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:35.809283 10271 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:35.810494  9238 consensus_peers.cc:597] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7cfb4241815c471a8b29031ba9674528. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:35.840382 10271 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.842811  9194 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.843377  9122 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:35.845139 10169 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:35.859587  9115 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.860953  9021 catalog_manager.cc:5697] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:35.932806  9270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:35.933498  9270 raft_consensus.cc:606] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:35.947310  9115 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:35.948078  9115 raft_consensus.cc:606] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:35.950752  9270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:35.951303  9270 raft_consensus.cc:606] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:35.966001  9194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:35.966514  9194 raft_consensus.cc:606] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:35.969214  9270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:35.969776  9270 raft_consensus.cc:606] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:35.972438  9270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c351454182cf49938eb92870b1fb297e"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:35.973012  9270 raft_consensus.cc:606] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:35.975571  9270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:35.976176  9270 raft_consensus.cc:606] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:35.977380  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:03:35.978979  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:36.017374 10317 raft_consensus.cc:993] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:36.017966 10317 raft_consensus.cc:1081] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:36.019701  9858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:03:36.020424  9858 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:36.020823  9858 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.027426  9858 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.029964  9858 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:36.030805  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:36.030892  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:36.031571  9115 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.032095  9270 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:36.032411  9270 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:36.033113  9270 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.034165  9270 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.038357  9115 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:36.038645  9270 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:36.039851  9825 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:36.041775 10324 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:36.043073 10324 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:36.044044 10324 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.056860  9018 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.097141 10219 raft_consensus.cc:993] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:36.097640 10297 raft_consensus.cc:1081] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:36.100594  9270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39754
I20260709 13:03:36.101082  9270 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:36.101428  9270 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.106673  9270 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:36.108791  9270 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:36.108980  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:36.109576  9122 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:36.109863  9122 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:36.110543  9122 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:36.111645  9122 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.112826  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:36.113395  9194 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.117820  9122 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:36.119457  9235 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:36.122316 10271 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:36.122931 10271 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:36.124099 10271 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:36.119518  9194 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:36.144464  9018 catalog_manager.cc:5697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.157181 10329 ts_tablet_manager.cc:933] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:36.159607 10329 tablet_copy_client.cc:323] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:36.161540  9207 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet 06c3356087cc45e78fa0f79168b68e8c from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:36.162146  9207 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet 06c3356087cc45e78fa0f79168b68e8c from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-06c3356087cc45e78fa0f79168b68e8c
I20260709 13:03:36.170449  9207 tablet_copy_source_session.cc:215] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:36.183279 10329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06c3356087cc45e78fa0f79168b68e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.199281 10327 raft_consensus.cc:993] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:36.199841 10271 raft_consensus.cc:1081] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:36.208927 10331 ts_tablet_manager.cc:933] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:36.212509  9194 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38858
I20260709 13:03:36.213186  9194 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:36.213482  9194 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.216862 10331 tablet_copy_client.cc:323] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:36.217748 10329 tablet_copy_client.cc:806] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:36.218276 10329 tablet_copy_client.cc:670] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:36.219636  9194 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:36.220185  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 7cfb4241815c471a8b29031ba9674528 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:36.220790  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 7cfb4241815c471a8b29031ba9674528 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-7cfb4241815c471a8b29031ba9674528
I20260709 13:03:36.223289  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:36.224045  9267 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:36.224373  9267 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:36.224941  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:36.225579  9115 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.225168  9267 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:36.226336  9267 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.229218  9282 tablet_copy_source_session.cc:215] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:36.234771  9115 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:36.235102 10329 tablet_copy_client.cc:538] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:36.236099  9161 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:36.237047 10304 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:36.237210  9267 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:36.237612 10304 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:36.240048 10331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cfb4241815c471a8b29031ba9674528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.240782 10304 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:36.244678 10329 tablet_bootstrap.cc:492] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:36.256469  9194 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:36.267346 10333 ts_tablet_manager.cc:933] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:36.288204 10333 tablet_copy_client.cc:323] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:36.290570  9207 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet b0917335c1234d5abec1e40c742de6be from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:36.291098  9207 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet b0917335c1234d5abec1e40c742de6be from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-b0917335c1234d5abec1e40c742de6be
I20260709 13:03:36.299168 10331 tablet_copy_client.cc:806] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:36.299858 10331 tablet_copy_client.cc:670] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:36.301803  9018 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.303922 10331 tablet_copy_client.cc:538] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:36.305428  9207 tablet_copy_source_session.cc:215] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:36.306056 10168 raft_consensus.cc:993] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:36.306910 10334 raft_consensus.cc:1081] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:36.309181  9858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:03:36.309952  9858 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:36.310429  9858 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.314528 10331 tablet_bootstrap.cc:492] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:36.331732 10327 raft_consensus.cc:993] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:36.333492 10169 raft_consensus.cc:1081] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:36.335785  9115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c351454182cf49938eb92870b1fb297e"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:03:36.336527  9115 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:36.337026  9115 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.344833 10327 raft_consensus.cc:993] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:36.345557 10169 raft_consensus.cc:1081] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:36.347896  9862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:03:36.348580  9862 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:36.349068  9862 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.350131 10331 tablet_bootstrap.cc:492] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:36.351333 10331 tablet_bootstrap.cc:492] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:36.352092 10331 ts_tablet_manager.cc:1403] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.014s
I20260709 13:03:36.353824  9115 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:36.363348  9858 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.374348  9858 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:36.371245  9862 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.364207 10333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0917335c1234d5abec1e40c742de6be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:36.380759  9862 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:36.381564  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:36.382207  9194 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.384720  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:36.385262  9190 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:36.385783  9190 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:36.385197  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:36.386917  9191 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.388789  9115 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:36.386600  9190 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.380602 10327 raft_consensus.cc:993] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:36.392628  9190 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.396670 10317 raft_consensus.cc:1081] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:36.412947  9194 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:36.418982  9191 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:36.422027  9089 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:36.427034 10219 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:36.427587 10219 raft_consensus.cc:697] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:36.428493 10219 consensus_queue.cc:237] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:36.429705  9828 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:36.432430 10324 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:36.433041 10324 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:36.433959 10324 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.435416 10329 tablet_bootstrap.cc:492] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:03:36.436685  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 60605us
I20260709 13:03:36.439126  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:36.439635  9267 raft_consensus.cc:3055] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:36.440006  9267 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:36.448710  9194 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:36.448761  9267 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:36.450842  9267 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.457060  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:36.457844  9270 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.475420  9267 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:36.476418  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:36.477051  9265 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:36.477401  9265 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:36.478116  9265 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.481231  9265 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.459808  9858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:03:36.485450 10329 tablet_bootstrap.cc:492] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:36.497620  9270 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:36.498015 10329 ts_tablet_manager.cc:1403] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.254s	user 0.047s	sys 0.035s
I20260709 13:03:36.506225  9265 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:36.514297  9267 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:36.516531  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:03:36.500242  9858 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:36.527868 10325 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:36.527002  9858 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.558287 10343 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:03:36.545265 10325 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:03:36.564955  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 87583.9us
I20260709 13:03:36.563676 10329 raft_consensus.cc:359] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:36.579959 10325 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.585793 10339 ts_tablet_manager.cc:933] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:36.576711 10329 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:36.594555 10329 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:36.598098 10329 ts_tablet_manager.cc:1434] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.097s	user 0.004s	sys 0.004s
I20260709 13:03:36.601270  9018 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.619541  9191 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:36.620749 10333 tablet_copy_client.cc:806] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:36.621400 10333 tablet_copy_client.cc:670] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:36.621444  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-06c3356087cc45e78fa0f79168b68e8c received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:36.621985  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-06c3356087cc45e78fa0f79168b68e8c on tablet 06c3356087cc45e78fa0f79168b68e8c with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:36.605719  9190 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:36.562104 10331 raft_consensus.cc:359] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:36.625329 10331 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:36.626627 10331 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:36.630004 10331 ts_tablet_manager.cc:1434] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.278s	user 0.009s	sys 0.004s
I20260709 13:03:36.632671 10339 tablet_copy_client.cc:323] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:36.634617  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet 709565930ea044f08322d94e29d83adf from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:36.635205  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet 709565930ea044f08322d94e29d83adf from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-709565930ea044f08322d94e29d83adf
I20260709 13:03:36.635933 10327 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:36.650401  9122 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:36.651078 10333 tablet_copy_client.cc:538] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:36.656867  9122 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:03:36.668116  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 146361us
I20260709 13:03:36.696544  9862 raft_consensus.cc:1217] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:36.702510  9021 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:36.705451  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-7cfb4241815c471a8b29031ba9674528 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:36.710992  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-7cfb4241815c471a8b29031ba9674528 on tablet 7cfb4241815c471a8b29031ba9674528 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:36.712459  9858 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.700984 10340 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:03:36.699609 10325 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:03:36.772428  9018 catalog_manager.cc:5697] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:36.778693  9862 raft_consensus.cc:1217] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:03:36.801111  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 92938.7us
I20260709 13:03:36.793447  9858 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:36.821166  9115 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:36.851119  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:36.852049  9122 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:36.859018  9122 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:36.840209 10356 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:03:36.875502  9265 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:36.880393  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43923.6us
I20260709 13:03:36.884523  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:36.893173  9133 tablet_copy_source_session.cc:215] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:36.907840 10333 tablet_bootstrap.cc:492] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:36.913437  9825 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:36.915926 10324 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:36.916497 10324 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:36.918417 10324 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:36.955132  9268 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:36.965155 10363 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:03:36.955619  9268 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:36.966919  9862 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:36.983676  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:36.993680 10339 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709565930ea044f08322d94e29d83adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:37.000339  9265 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:37.008551 10324 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:37.021155  9020 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:37.030637  9265 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:37.033974  9268 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:37.041162 10300 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:37.036091  9268 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:37.099900 10365 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:37.102154  9196 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:37.113121 10344 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:03:37.123274  9196 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:37.138480  9268 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
W20260709 13:03:37.163023 10366 log.cc:927] Time spent T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.151s	user 0.002s	sys 0.000s
I20260709 13:03:37.199880  9267 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:37.192544 10340 raft_consensus.cc:1064] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:37.192358 10324 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
W20260709 13:03:37.253086  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.282078 10340 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:37.288574 10344 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
W20260709 13:03:37.360083  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 390239us
W20260709 13:03:37.036460 10353 consensus_queue.cc:1175] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "92919b70789041c8a0de0b3525154fd1" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:03:37.361639 10371 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:03:37.405057 10339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40106 (request call id 245) took 1053 ms. Trace:
I20260709 13:03:37.405556 10339 rpcz_store.cc:276] 0709 13:03:36.351882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:36.352083 (+   201us) service_pool.cc:224] Handling call
0709 13:03:36.586297 (+234214us) ts_tablet_manager.cc:934] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
0709 13:03:36.990658 (+404361us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:03:37.188202 (+197544us) tablet_metadata.cc:665] Metadata flushed
0709 13:03:37.189451 (+  1249us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:03:37.375595 (+186144us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:03:37.399115 (+ 23520us) tablet_replica.cc:206] Creating consensus instance
0709 13:03:37.404902 (+  5787us) ts_tablet_manager.cc:989] Replica registered.
0709 13:03:37.404979 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":288,"tablet-copy.queue_time_us":158748,"thread_start_us":132000,"threads_started":2}
I20260709 13:03:37.407133 10339 tablet_copy_client.cc:806] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:37.407763  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:37.407687 10339 tablet_copy_client.cc:670] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
W20260709 13:03:37.500995 10377 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:03:37.502462 10333 log.cc:927] Time spent T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:03:37.521823 10339 tablet_copy_client.cc:538] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:37.534356  9190 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:37.540189  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 117656us
I20260709 13:03:37.561610 10334 raft_consensus.cc:1064] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:37.569783  9121 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:37.570755 10333 tablet_bootstrap.cc:492] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:37.571550 10344 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:37.574460 10334 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:37.578380 10333 tablet_bootstrap.cc:492] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:37.583956  9122 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:37.591533  9860 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:37.599143 10383 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:37.592422 10333 ts_tablet_manager.cc:1403] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.685s	user 0.027s	sys 0.026s
I20260709 13:03:37.658319 10339 tablet_bootstrap.cc:492] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
W20260709 13:03:37.704504  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 198920us
W20260709 13:03:37.715251  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:37.753186 10219 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:37.741291 10381 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:03:37.808868  9270 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:37.826285 10384 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:03:37.797794 10333 raft_consensus.cc:359] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:37.853401 10333 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:37.868427  9857 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:37.868666 10333 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:37.907887 10333 ts_tablet_manager.cc:1434] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.286s	user 0.009s	sys 0.000s
W20260709 13:03:37.918835 10387 log.cc:927] Time spent T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.263s	user 0.004s	sys 0.000s
W20260709 13:03:37.926218  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 172681us
W20260709 13:03:37.925748 10388 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.201s	user 0.004s	sys 0.002s
W20260709 13:03:37.989871  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 231328us
W20260709 13:03:38.042150  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 291474us
I20260709 13:03:38.091224 10325 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:38.134326 10325 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:38.164448  9194 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:38.165628  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-b0917335c1234d5abec1e40c742de6be received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:38.166126  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-b0917335c1234d5abec1e40c742de6be on tablet b0917335c1234d5abec1e40c742de6be with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:38.168756 10325 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:38.199472 10401 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:03:38.206182 10398 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:38.206969 10398 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:38.209138 10398 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:38.240483  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:38.250818  9114 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:38.257429 10294 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:38.262212  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:38.273307 10327 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:38.247772 10340 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:38.287245  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:38.290431  9195 raft_consensus.cc:606] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:38.306914  9268 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:38.289181 10340 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:38.359002 10340 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:38.343370 10401 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:38.372407 10401 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:38.387207 10373 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:03:38.389773 10364 raft_consensus.cc:993] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:38.390514 10364 raft_consensus.cc:1081] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:38.462641 10373 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:38.463203 10373 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:38.469172  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:38.470906  9828 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:38.472056 10410 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:38.484571  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 218873us
I20260709 13:03:38.489080 10401 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:38.493239 10381 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:38.493978 10381 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:38.496728 10381 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:38.508008 10373 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:38.514008  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:38.517777  9857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:03:38.518410  9857 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:38.518772  9857 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.523455  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:38.526508  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:38.526865  9121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49752620cbbf430991915a2400f71286"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:38.527657  9121 raft_consensus.cc:606] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:38.542583  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:38.545676 10324 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:38.550252  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:38.527559  9857 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:38.555817  9857 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:38.556720  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:38.557529  9189 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:38.557898  9189 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:38.558583  9861 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:38.558787  9189 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:38.559794  9189 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.565444 10382 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.567122  9189 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:38.568305  9828 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:38.569154 10324 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:38.569684 10324 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:38.570528 10324 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:38.582139  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:38.588136 10339 tablet_bootstrap.cc:492] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:38.590804 10339 tablet_bootstrap.cc:492] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:38.591500 10339 ts_tablet_manager.cc:1403] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.933s	user 0.050s	sys 0.056s
I20260709 13:03:38.594338 10339 raft_consensus.cc:359] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:38.595142 10339 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:38.595533  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:38.595832 10339 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:38.598619 10339 ts_tablet_manager.cc:1434] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:03:38.600421  9189 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:38.600962  9189 raft_consensus.cc:606] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:38.601909  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-709565930ea044f08322d94e29d83adf received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:38.602341  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-709565930ea044f08322d94e29d83adf on tablet 709565930ea044f08322d94e29d83adf with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:38.634022  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:38.638171 10271 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:38.642130  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:38.643667  9264 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.649618  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:38.650190  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:38.652750  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:38.660202  9189 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "991b28944986479a8ad096ca92f1a435"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:38.661193  9189 raft_consensus.cc:606] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:38.662439  9114 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:38.655293 10403 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:38.665939  9162 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:38.666956 10168 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:38.670580  9020 catalog_manager.cc:5697] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:38.671814  9272 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:38.692727  9269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:38.693389  9269 raft_consensus.cc:606] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:38.699805 10356 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.718413 10364 raft_consensus.cc:993] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:38.720711 10404 raft_consensus.cc:1081] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:38.722221  9270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:38.740840  9270 raft_consensus.cc:606] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:38.764557  9264 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:38.763861  9858 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:03:38.773627  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 58440us
I20260709 13:03:38.773720  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:03:38.778862  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:38.783066  9270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "991b28944986479a8ad096ca92f1a435"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:38.785938  9270 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:38.786482  9270 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.784868  9862 raft_consensus.cc:1217] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:38.810632  9862 raft_consensus.cc:1217] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:38.811599  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:38.818640  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:38.820487 10327 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:38.815241  9018 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:38.831116  9270 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:38.836138  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:38.836144  9862 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:38.838922  9121 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.839208  9114 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:38.841926  9270 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:38.856547  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:38.862654  9190 raft_consensus.cc:3055] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:38.863039  9190 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:38.863879  9190 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:38.865342  9190 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.865693 10399 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:38.872115 10357 raft_consensus.cc:993] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:38.872711 10357 raft_consensus.cc:1081] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:38.888940 10404 raft_consensus.cc:993] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:38.889469 10294 raft_consensus.cc:1081] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:38.894850  9121 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:38.901292 10382 raft_consensus.cc:993] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:38.901819 10382 raft_consensus.cc:1081] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
W20260709 13:03:38.909852  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48729.5us
W20260709 13:03:38.909896  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 49578.9us
I20260709 13:03:38.914728  9862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:03:38.915355  9862 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:38.916107  9862 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:38.919179  9190 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:38.920290  9858 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:03:38.920959  9858 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:38.921371  9858 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.929348  9235 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:38.928738  9268 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49752620cbbf430991915a2400f71286"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39754
I20260709 13:03:38.934902  9268 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:38.935431  9268 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.941419 10399 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:38.942060 10399 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:38.942986 10399 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:38.944366  9858 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:38.947969  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:38.948791  9191 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:38.949142  9191 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:38.950150  9191 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:38.951608  9191 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.960827  9862 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:38.975767  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:38.976672  9194 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:38.974403  9862 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:38.980103  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:38.980813  9121 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:38.982959  9191 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:38.984414  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:38.987349 10324 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:38.994596 10428 raft_consensus.cc:993] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:38.994602  9194 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:38.995363 10428 raft_consensus.cc:1081] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:38.995469  9268 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:38.996161  9828 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:38.970448  9858 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:38.987895 10324 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:38.997884  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:38.986835  9197 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:39.004004  9264 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:39.004967  9862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:03:39.005481 10410 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:39.006111  9862 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:39.005582 10324 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:39.006541  9862 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:39.004735  9264 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:39.006404 10410 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:39.007721  9264 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:39.009635  9264 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:39.009385 10410 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:39.010191  9268 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:39.069307  9121 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:39.079023  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:39.093362  9191 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:39.095100  9021 catalog_manager.cc:5697] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:39.138628  9862 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:39.148378  9862 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:39.150406  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:39.151407  9268 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:39.151790  9268 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:39.153173  9268 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:39.155659  9268 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:39.156333  9264 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:39.165275  9197 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:39.176389  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:39.183593  9191 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:39.187471  9264 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:39.206272  9121 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:39.206738  9121 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:39.214874 10410 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:39.236122 10394 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:39.248551 10394 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:39.260401  9268 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:39.263289  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:39.264688 10340 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:39.265347 10340 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:39.274518 10394 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:39.266641 10340 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:39.286887  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:39.291087  9270 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:39.293293 10401 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:39.309687  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:03:39.312376 10437 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:39.312966 10437 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:39.332770  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:39.333779  9119 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:39.314111 10437 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:39.372685  9858 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:39.374042  9264 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:39.331595  9121 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:39.377550  9121 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:39.374542 10401 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:39.388831  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:39.399423  9190 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:39.405009  9193 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:39.435101  9269 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:39.457154  9121 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:39.459487 10373 raft_consensus.cc:1064] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:39.447111  9119 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:39.443864 10367 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:03:39.462391 10373 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:39.439706 10394 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
W20260709 13:03:39.468864  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44793.1us
I20260709 13:03:39.492206  9119 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:39.495393  9235 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:39.497304  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 06c3356087cc45e78fa0f79168b68e8c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:39.492431  9859 raft_consensus.cc:2955] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:39.515857 10437 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:39.516187  9270 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:39.511674 10383 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:39.529184 10410 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:39.545120 10410 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:39.556628  9121 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:39.576365  9018 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 3 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:39.578399 10406 raft_consensus.cc:1064] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:39.578717  9018 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:39.589637 10327 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:03:39.594592  9197 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:39.600857 10440 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:39.606942 10406 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:39.612536  9017 catalog_manager.cc:5697] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:39.628933  9860 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:39.629652  9017 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:39.633773 10367 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:39.636066  9121 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:39.690987  9191 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:39.718640 10325 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:03:39.780074  9193 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:39.823066 10383 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:03:39.832055  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 63966.4us
I20260709 13:03:39.834971 10404 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:03:39.852823  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:39.861474  9265 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:39.916692  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 117438us
I20260709 13:03:39.932355  9119 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:39.929543  9017 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:39.935544  9860 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:39.947316  9017 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:40.031865  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 205144us
I20260709 13:03:40.065377  9120 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:40.066831  9268 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:40.069377  9269 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:40.079043  9192 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:40.111164 10404 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.113963 10367 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:40.116484 10410 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:40.101622 10427 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:40.063841 10347 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:40.131947  9121 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.144059  9857 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.151371 10406 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:03:40.172619  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 97344us
I20260709 13:03:40.173589 10342 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:40.174247 10342 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:40.176162  9268 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:40.183136  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:40.185986 10437 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:40.188426  9862 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:40.180856 10342 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:40.204849 10460 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:40.214540  9269 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.221952  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:40.231050 10342 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:40.233592  9193 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:40.232012 10460 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.238713  9087 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:40.239527 10347 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:40.241498  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:40.244441 10327 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:40.250725  9861 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.260540 10406 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.270618  9193 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.282297  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet 06c3356087cc45e78fa0f79168b68e8c with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:40.288230  9861 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.300920  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7cfb4241815c471a8b29031ba9674528 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:40.322791 10480 tablet_replica.cc:333] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:40.324539 10480 raft_consensus.cc:2243] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:40.325497 10480 raft_consensus.cc:2272] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:40.329054 10480 ts_tablet_manager.cc:1916] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:40.343465  9020 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.346005  9020 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.368431 10480 ts_tablet_manager.cc:1929] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:40.369247 10480 log.cc:1199] T 06c3356087cc45e78fa0f79168b68e8c P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/06c3356087cc45e78fa0f79168b68e8c
I20260709 13:03:40.368350  9021 catalog_manager.cc:5697] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.371495  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet 06c3356087cc45e78fa0f79168b68e8c (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:40.375680  9192 raft_consensus.cc:2955] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.404431  9268 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.409955  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet 7cfb4241815c471a8b29031ba9674528 with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:40.411237 10480 tablet_replica.cc:333] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:40.412305 10480 raft_consensus.cc:2243] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:40.413112 10480 raft_consensus.cc:2272] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:40.414220  9121 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:40.415987 10480 ts_tablet_manager.cc:1916] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:40.403599  9195 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:40.424826  9861 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:40.429117 10347 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:40.429924  9267 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:40.429430  9861 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:40.441483 10357 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:03:40.436584 10415 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:03:40.444756  9121 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:40.447450 10406 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.456703 10480 ts_tablet_manager.cc:1929] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:40.457278 10480 log.cc:1199] T 7cfb4241815c471a8b29031ba9674528 P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/7cfb4241815c471a8b29031ba9674528
I20260709 13:03:40.460045  9268 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.459795  9861 raft_consensus.cc:2955] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.471596 10465 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.485258  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 709565930ea044f08322d94e29d83adf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:40.485736  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b0917335c1234d5abec1e40c742de6be with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:40.485966  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet 7cfb4241815c471a8b29031ba9674528 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:40.486936 10415 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:40.494596  9857 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.496109  9018 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:40.510581  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet b0917335c1234d5abec1e40c742de6be with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:40.512332 10481 tablet_replica.cc:333] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:40.514611 10481 raft_consensus.cc:2243] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:40.515334 10481 raft_consensus.cc:2272] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:40.560065  9119 raft_consensus.cc:2955] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.621493  9021 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:40.636802  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:40.643188 10481 ts_tablet_manager.cc:1916] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:40.731151 10481 ts_tablet_manager.cc:1929] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:40.731662 10481 log.cc:1199] T b0917335c1234d5abec1e40c742de6be P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/b0917335c1234d5abec1e40c742de6be
I20260709 13:03:40.735487  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet b0917335c1234d5abec1e40c742de6be (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:40.754359  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet 709565930ea044f08322d94e29d83adf with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:40.790452 10486 tablet_replica.cc:333] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:40.798894 10486 raft_consensus.cc:2243] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:40.799672 10486 raft_consensus.cc:2272] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:40.802862 10486 ts_tablet_manager.cc:1916] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:40.818302  9269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:40.819008  9269 raft_consensus.cc:606] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:40.820441 10486 ts_tablet_manager.cc:1929] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:40.820927 10486 log.cc:1199] T 709565930ea044f08322d94e29d83adf P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/709565930ea044f08322d94e29d83adf
I20260709 13:03:40.821897  9120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:40.822400  9120 raft_consensus.cc:606] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:40.822567  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet 709565930ea044f08322d94e29d83adf (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:40.825066  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:40.825626  9195 raft_consensus.cc:606] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:40.828171  9120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a35f9598c50046539a7f41287876135d"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:40.828709  9120 raft_consensus.cc:606] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:40.830986  9269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:40.831447  9269 raft_consensus.cc:606] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:40.832429  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:03:40.833488  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:40.965278 10219 raft_consensus.cc:993] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:40.965837 10441 raft_consensus.cc:1081] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:40.967159  9195 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a35f9598c50046539a7f41287876135d"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38874
I20260709 13:03:40.967849  9195 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:40.968214  9195 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:40.973881  9195 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:40.975620  9195 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:40.976325  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:40.976611  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:40.976969  9120 raft_consensus.cc:3055] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:40.977267  9861 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:40.977337  9120 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:40.978264  9120 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:40.979619  9120 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:40.983258  9861 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:40.984522  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:40.985841 10485 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:40.986356 10485 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:40.988461 10485 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:40.986065  9120 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:41.001485  9018 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.152024 10484 raft_consensus.cc:993] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:41.152470 10485 raft_consensus.cc:1081] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:41.154157  9268 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:41.154796  9268 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:41.155206  9268 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.161758  9268 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:41.163408  9268 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:41.165413  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:41.166031  9861 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.166066  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:41.166831  9195 raft_consensus.cc:3055] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:41.167181  9195 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:41.168021  9195 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:41.169275  9195 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.177022  9195 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:41.177812  9861 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:41.178503  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:41.179487 10382 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:41.180060 10382 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:41.181061 10382 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:41.196375  9018 catalog_manager.cc:5697] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.207144 10427 raft_consensus.cc:993] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:41.207654 10327 raft_consensus.cc:1081] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:41.209134  9195 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38858
I20260709 13:03:41.209612  9195 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:41.209995  9195 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.215641 10427 raft_consensus.cc:993] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:41.216226 10427 raft_consensus.cc:1081] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:41.218592  9861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:03:41.219559  9861 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:41.218878  9195 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:41.220041  9861 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.222113  9195 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:41.223929  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:41.224633  9119 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.226724  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:41.227347  9264 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:41.227029  9861 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:41.227640  9264 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:41.228539  9264 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:41.229656  9264 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.230387  9861 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:41.230598  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:41.231218  9120 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.231480  9119 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:41.232133  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:41.232731  9269 raft_consensus.cc:3055] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:41.233039  9269 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:41.233124  9161 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:41.234180 10484 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:41.234742 10484 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:41.233933  9269 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:41.237568  9269 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.235615 10484 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:41.248392  9269 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:41.249390  9264 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:41.250720  9826 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:03:41.251695 10437 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:41.252296 10437 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:41.253386 10437 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:41.240114  9120 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:41.264027  9018 catalog_manager.cc:5697] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:41.268901 10441 raft_consensus.cc:993] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:41.269564 10441 raft_consensus.cc:1081] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:41.273034  9017 catalog_manager.cc:5697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:41.274345  9267 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39754
I20260709 13:03:41.274907  9267 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:41.275318  9267 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.282248  9267 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:41.284703  9267 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:41.285244  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:41.285677  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:41.285987  9120 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:41.286288  9193 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.286453  9120 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:41.287374  9120 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:41.288508  9120 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:41.293493  9193 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:41.294942  9238 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:41.295625 10327 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:41.296120 10327 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:41.296995 10327 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:41.296120  9120 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:41.315733  9017 catalog_manager.cc:5697] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:41.399782  9857 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:41.400930 10485 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:41.412707  9120 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:41.413980 10485 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:41.645964  9267 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:41.647120 10485 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:41.656749  9120 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:41.658650 10415 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:41.664548  9264 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:41.666136 10439 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:41.689383  9119 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:41.692410 10439 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:41.712865  9195 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:41.715866 10427 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:41.732918  9195 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:41.736490 10427 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:41.744051  9857 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:41.745462 10427 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:41.754896  9119 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:41.756664 10327 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:41.834342  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:41.873198  9121 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:41.880781  9089 consensus_peers.cc:597] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 470326df6cae476c90888533e504e6ec. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:41.881297  9267 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:41.882411  9195 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:41.882616 10357 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:41.883576 10219 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:41.884555  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:41.896611 10219 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:41.905360  9195 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:41.922140  9267 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:41.935204  9017 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:41.948022  9195 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:41.955565  9164 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a3eaf34ab444dca94a52a6fce2a556e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:41.956354  9267 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:41.957417  9120 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:41.958014 10484 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:41.960259 10415 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:41.972400  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:41.973038  9193 raft_consensus.cc:606] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:41.975426 10415 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:41.980798  9267 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:41.983156  9121 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:41.987617  9119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "470326df6cae476c90888533e504e6ec"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:41.988224  9119 raft_consensus.cc:606] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:42.000069  9264 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:42.000910  9264 raft_consensus.cc:606] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:42.005779  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:03:42.007378  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:42.008740  9018 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:42.005301  9195 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.020402 10506 raft_consensus.cc:993] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:42.021037 10506 raft_consensus.cc:1081] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:42.023290  9120 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:03:42.023895  9120 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:42.024259  9120 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:42.032636  9164 consensus_peers.cc:597] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e9a6ffaa328644f4bd7f129f8fa22875. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:42.033725  9121 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:42.037389  9264 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:42.038013 10518 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:42.040536 10484 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:42.046681  9120 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:42.048966  9120 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:42.049602  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:42.049878  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:42.050151  9195 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:42.050316  9267 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:42.050592  9267 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:42.051185  9267 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:42.051978  9267 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:42.048534  9119 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:42.056995  9264 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:42.058290  9267 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:42.059980  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:42.060830 10510 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:42.046725 10518 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:42.062175 10510 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:42.056996  9195 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:42.063256 10510 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:42.080986  9019 catalog_manager.cc:5697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.091598  9120 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.097652  9019 catalog_manager.cc:5697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.112326  9264 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:42.114148  9193 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:42.115512 10511 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:42.117769 10511 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:42.120939  9089 consensus_peers.cc:597] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a51a0795b52e4843959cd3be3d077713. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:42.128080 10510 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:42.132367  9264 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:42.143687  9193 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:42.162114  9018 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:42.320420 10524 ts_tablet_manager.cc:933] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:42.325877 10524 tablet_copy_client.cc:323] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:42.327824  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet 470326df6cae476c90888533e504e6ec from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:42.328281  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet 470326df6cae476c90888533e504e6ec from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-470326df6cae476c90888533e504e6ec
I20260709 13:03:42.334674  9133 tablet_copy_source_session.cc:215] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:42.337733 10524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 470326df6cae476c90888533e504e6ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:42.340893 10511 raft_consensus.cc:993] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:42.341408 10511 raft_consensus.cc:1081] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:42.345743  9193 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "470326df6cae476c90888533e504e6ec"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38874
I20260709 13:03:42.346316  9193 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:42.346680  9193 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:42.350951  9193 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.359378  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:42.360073  9119 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:42.360376  9119 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:42.360451  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:42.361017  9264 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:42.361166  9119 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.362818 10524 tablet_copy_client.cc:806] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:42.362432  9119 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:42.363433 10524 tablet_copy_client.cc:670] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:42.359530  9193 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:42.368134  9264 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:42.369300 10524 tablet_copy_client.cc:538] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:42.369375  9162 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:42.370363 10415 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:42.370842 10415 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:42.371680 10415 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.380290 10524 tablet_bootstrap.cc:492] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:42.396078 10518 raft_consensus.cc:993] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:42.396615 10518 raft_consensus.cc:1081] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:42.398933  9267 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:42.399454  9267 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:42.399657  9119 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:42.399971  9267 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:42.387158  9019 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.406620  9267 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:42.411243  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:42.411789  9267 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:42.411916  9192 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:42.412338  9192 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:42.413074  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:42.413643  9861 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:42.413097  9192 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:42.416003  9192 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:42.416407 10524 tablet_bootstrap.cc:492] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:42.417351 10524 tablet_bootstrap.cc:492] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:42.417632  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "470326df6cae476c90888533e504e6ec"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:42.418179  9193 raft_consensus.cc:606] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:03:42.418310 10524 ts_tablet_manager.cc:1403] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.009s
I20260709 13:03:42.420349  9861 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:42.421486  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:03:42.421700 10524 raft_consensus.cc:359] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.422235 10427 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:42.422621  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "470326df6cae476c90888533e504e6ec"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:42.422904 10427 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:42.423209  9193 raft_consensus.cc:606] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:03:42.424122 10524 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:42.424155 10427 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:03:42.425204  9008 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 470326df6cae476c90888533e504e6ec already in progress [suppressed 1 similar messages]
I20260709 13:03:42.424934 10524 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.427772 10524 ts_tablet_manager.cc:1434] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:03:42.430779  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-470326df6cae476c90888533e504e6ec received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:42.431404  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-470326df6cae476c90888533e504e6ec on tablet 470326df6cae476c90888533e504e6ec with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:42.439159  9192 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:42.439911  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "470326df6cae476c90888533e504e6ec"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:42.440488  9193 raft_consensus.cc:606] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:03:42.444398  9019 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:42.445220  9069 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:03:42.445688  9069 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 470326df6cae476c90888533e504e6ec already in progress
W20260709 13:03:42.446012  9069 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 470326df6cae476c90888533e504e6ec already in progress
I20260709 13:03:42.497835 10524 ts_tablet_manager.cc:933] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:42.511307 10524 tablet_copy_client.cc:323] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:42.513039  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet a51a0795b52e4843959cd3be3d077713 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:42.513592  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet a51a0795b52e4843959cd3be3d077713 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-a51a0795b52e4843959cd3be3d077713
I20260709 13:03:42.517468  9264 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:42.519546 10510 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:42.521139  9133 tablet_copy_source_session.cc:215] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:42.526677 10524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a51a0795b52e4843959cd3be3d077713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:42.536432  9193 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:42.539271 10511 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:42.547679 10524 tablet_copy_client.cc:806] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:42.548530 10524 tablet_copy_client.cc:670] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:42.552829 10524 tablet_copy_client.cc:538] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:42.553265 10533 ts_tablet_manager.cc:933] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:42.561018 10533 tablet_copy_client.cc:323] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:42.563364  9207 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet 9a3eaf34ab444dca94a52a6fce2a556e from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:42.563985  9207 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet 9a3eaf34ab444dca94a52a6fce2a556e from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-9a3eaf34ab444dca94a52a6fce2a556e
I20260709 13:03:42.569284 10524 tablet_bootstrap.cc:492] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:42.636003  9207 tablet_copy_source_session.cc:215] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:42.641467 10533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a3eaf34ab444dca94a52a6fce2a556e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:42.667801 10533 tablet_copy_client.cc:806] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:42.668628 10533 tablet_copy_client.cc:670] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:42.673053 10533 tablet_copy_client.cc:538] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:42.704219 10539 ts_tablet_manager.cc:933] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:42.704718 10524 tablet_bootstrap.cc:492] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:42.707360 10524 tablet_bootstrap.cc:492] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:42.708092 10524 ts_tablet_manager.cc:1403] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.139s	user 0.026s	sys 0.011s
I20260709 13:03:42.712947 10533 tablet_bootstrap.cc:492] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:42.713903 10539 tablet_copy_client.cc:323] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:42.715664  9207 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet e9a6ffaa328644f4bd7f129f8fa22875 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:42.716207  9207 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet e9a6ffaa328644f4bd7f129f8fa22875 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-e9a6ffaa328644f4bd7f129f8fa22875
I20260709 13:03:42.719260 10524 raft_consensus.cc:359] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.731969 10524 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:42.732901 10524 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.735458 10524 ts_tablet_manager.cc:1434] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:03:42.736558  9207 tablet_copy_source_session.cc:215] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:42.750638  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-a51a0795b52e4843959cd3be3d077713 received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:42.751119  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-a51a0795b52e4843959cd3be3d077713 on tablet a51a0795b52e4843959cd3be3d077713 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:42.781062 10539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9a6ffaa328644f4bd7f129f8fa22875. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:42.822889  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 61547.5us
I20260709 13:03:42.826508  9861 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Advancing to term 2
I20260709 13:03:42.832754 10533 tablet_bootstrap.cc:492] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:42.837710 10533 tablet_bootstrap.cc:492] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:42.838327 10533 ts_tablet_manager.cc:1403] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.126s	user 0.044s	sys 0.028s
I20260709 13:03:42.841338 10533 raft_consensus.cc:359] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.841981 10533 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:42.842579 10533 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.845111 10533 ts_tablet_manager.cc:1434] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:03:42.846110  9119 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:42.847456 10545 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:42.848317  9861 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:42.849834 10545 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:42.852241  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-9a3eaf34ab444dca94a52a6fce2a556e received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:42.852696  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-9a3eaf34ab444dca94a52a6fce2a556e on tablet 9a3eaf34ab444dca94a52a6fce2a556e with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:42.853814 10484 raft_consensus.cc:993] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:42.854377 10484 raft_consensus.cc:1081] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:42.859522  9264 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:42.859700  9119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "470326df6cae476c90888533e504e6ec"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43782
I20260709 13:03:42.860311  9119 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:42.860653  9119 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:42.861013 10518 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:42.880012  9119 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.882062 10518 raft_consensus.cc:1064] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
W20260709 13:03:42.882583 10518 raft_consensus.cc:1068] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: Unable to promote non-voter 92919b70789041c8a0de0b3525154fd1: Service unavailable: leader transfer in progress
I20260709 13:03:42.884320 10539 tablet_copy_client.cc:806] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:42.884881 10539 tablet_copy_client.cc:670] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:42.887349  9119 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:42.890426 10539 tablet_copy_client.cc:538] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:42.894308  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:42.894891  9265 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:42.910039 10539 tablet_bootstrap.cc:492] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:42.920240  9861 raft_consensus.cc:1217] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:42.926124  9265 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:03:42.927402  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:42.928277 10219 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:42.928433  9192 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:42.928764 10219 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:42.929666 10219 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.935236  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:42.936491  9192 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:42.936874  9192 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:42.937759  9192 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:42.939244  9192 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:42.943888 10427 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:42.968560  9192 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:03:42.980537  9018 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 3, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:42.985035  9862 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:42.987174 10538 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:43.008551  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:43.022722  9861 raft_consensus.cc:1217] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:43.034094 10539 tablet_bootstrap.cc:492] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:43.035028 10539 tablet_bootstrap.cc:492] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:43.036118 10539 ts_tablet_manager.cc:1403] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.126s	user 0.026s	sys 0.020s
I20260709 13:03:43.039556 10539 raft_consensus.cc:359] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:43.040396 10539 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:43.040992 10539 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:43.043393 10539 ts_tablet_manager.cc:1434] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:03:43.048089  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-e9a6ffaa328644f4bd7f129f8fa22875 received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:43.048550  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-e9a6ffaa328644f4bd7f129f8fa22875 on tablet e9a6ffaa328644f4bd7f129f8fa22875 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:43.114385  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:43.165019  9264 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:43.165624  9264 raft_consensus.cc:606] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:43.168488  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:43.169068  9195 raft_consensus.cc:606] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:43.171914  9119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:43.172451  9119 raft_consensus.cc:606] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:43.175297  9119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:43.175902  9119 raft_consensus.cc:606] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:43.177177  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:03:43.178656  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:43.205402 10219 raft_consensus.cc:993] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:43.205986 10542 raft_consensus.cc:1081] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:43.207608  9195 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38874
I20260709 13:03:43.208163  9195 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:43.208495  9195 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.213099  9195 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:43.215102  9195 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:43.216553  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:43.217084  9119 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:43.217403  9119 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:43.216086  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:43.218097  9264 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.218410  9119 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:43.219805  9119 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.225670  9119 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:43.226719  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:43.226768  9264 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:43.227385 10518 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:43.227931 10518 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:43.228785 10518 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:43.241999  9020 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.243973  9862 raft_consensus.cc:1217] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:03:43.370604  9195 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:43.372076 10542 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:43.377578  9861 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Advancing to term 3
I20260709 13:03:43.383322  9265 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:43.385337 10510 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:43.389947  9861 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:43.415086 10219 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:43.433786 10415 raft_consensus.cc:1064] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
W20260709 13:03:43.434267 10415 raft_consensus.cc:1068] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Unable to promote non-voter 92919b70789041c8a0de0b3525154fd1: Service unavailable: leader transfer in progress
I20260709 13:03:43.436389 10510 raft_consensus.cc:1064] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:43.438953 10510 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.449909  9862 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:43.451779 10219 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:43.456545  9265 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:43.457930 10219 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:43.461957  9192 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:43.463758 10510 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:43.468040 10327 raft_consensus.cc:993] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:43.468477 10427 raft_consensus.cc:1081] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:43.471174  9192 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38858
I20260709 13:03:43.471661  9192 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:43.470739 10542 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.472302  9192 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:43.480589  9195 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.484203  9265 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.487283 10557 raft_consensus.cc:993] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:43.487825 10557 raft_consensus.cc:1081] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:43.494230  9862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40106
I20260709 13:03:43.494776  9862 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:43.495108  9862 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.498027  9192 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.500826  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:43.501363  9264 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:43.501668  9264 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:43.502355  9264 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.503403  9264 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:43.508059  9857 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.516512  9264 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:43.517656  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:43.519553 10415 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:43.520272 10415 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:43.521039 10415 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.505682  9862 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.523785  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:43.524340  9120 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:43.524608  9120 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:43.517656  9017 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:43.525362  9120 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.527544  9120 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.534535  9120 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:43.536942  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:43.537367  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:43.537542  9193 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.538218 10523 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:43.538661 10523 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:43.539610 10523 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.544101  9193 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:43.545104  9862 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:43.565138  9192 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:43.568105  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:43.568778  9862 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:43.579281  9020 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.582022  9119 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.595072  9018 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:43.596571  9862 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:43.608130  9264 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:43.609402 10542 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:43.612216  9862 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:43.613413 10542 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:43.615269 10415 raft_consensus.cc:993] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:43.615864 10545 raft_consensus.cc:1081] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:43.619282 10510 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.621272  9265 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:43.621825  9265 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:43.622136  9265 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.626864  9861 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.631202  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 470326df6cae476c90888533e504e6ec with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:43.631114  9265 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:43.633971  9265 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:43.634727  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:43.636251  9119 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.647708  9119 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:43.648777  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:43.648947  9235 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:43.649381  9195 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:43.649689  9195 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:43.650015 10538 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:43.650486 10538 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:43.650369  9195 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:43.652070  9195 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:43.651435  9017 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 5 to 6, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.651389 10538 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:43.666664  9119 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:43.644364  9264 raft_consensus.cc:2955] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.670053 10415 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:43.671157  9195 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:43.674301  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet 470326df6cae476c90888533e504e6ec with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:43.681790 10562 tablet_replica.cc:333] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:43.690521  9862 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Advancing to term 2
I20260709 13:03:43.693502  9017 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:43.699381  9862 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:43.705451  9265 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:43.709821 10415 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:43.710503 10484 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:43.726063 10518 raft_consensus.cc:1064] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:43.729416  9264 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:43.730268  9264 raft_consensus.cc:606] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Received request to transfer leadership
I20260709 13:03:43.731537 10518 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.748414  9264 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:43.748967  9264 raft_consensus.cc:606] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Received request to transfer leadership
I20260709 13:03:43.761250  9862 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:43.764896  9264 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:43.767314 10415 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:43.771575 10567 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:43.774778 10568 raft_consensus.cc:1064] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:43.775442  9120 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 13:03:43.776962 10567 consensus_queue.cc:1175] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "92919b70789041c8a0de0b3525154fd1" responder_term: 2 status { last_received { term: 2 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 2 index: 3 } }
I20260709 13:03:43.782196 10562 raft_consensus.cc:2243] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:43.782977 10562 raft_consensus.cc:2272] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:43.787086 10562 ts_tablet_manager.cc:1916] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:43.795776 10566 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:43.805938  9264 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:43.806531  9264 raft_consensus.cc:606] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Received request to transfer leadership
W20260709 13:03:43.811371  9069 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:03:43.811829  9069 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 9a3eaf34ab444dca94a52a6fce2a556e already in progress
W20260709 13:03:43.812181  9069 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 9a3eaf34ab444dca94a52a6fce2a556e already in progress
I20260709 13:03:43.826614 10562 ts_tablet_manager.cc:1929] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
W20260709 13:03:43.829490  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56473.5us
I20260709 13:03:43.833068 10568 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.828253 10562 log.cc:1199] T 470326df6cae476c90888533e504e6ec P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/470326df6cae476c90888533e504e6ec
I20260709 13:03:43.838572  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet 470326df6cae476c90888533e504e6ec (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:43.843057  9862 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:43.844547 10545 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:43.856021  9265 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:43.856935  9120 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:43.857498 10568 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:43.860411 10566 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:43.881412 10415 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.884443  9861 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.889552 10567 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.897193  9119 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.902441  9861 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.902652  9018 catalog_manager.cc:5697] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:03:43.906409 10575 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:03:43.910915  9264 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.919869  9269 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.932860  9018 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.925302  9120 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.961012  9193 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.965435  9195 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:43.968853  9264 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:43.969473  9861 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:43.971402 10484 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:43.973634  9861 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:43.975708 10545 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:43.978641 10567 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:43.983065 10484 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.986847 10572 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.989657  9119 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:43.990783  9861 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.988492  9862 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:43.997159  9269 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:44.003312 10415 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:44.006181 10568 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:44.015074  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e9a6ffaa328644f4bd7f129f8fa22875 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:44.013188  9119 raft_consensus.cc:2955] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.018831  9018 catalog_manager.cc:5697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 4 to 5, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.024489  9193 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:44.025243  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a51a0795b52e4843959cd3be3d077713 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:44.025854 10559 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:44.026698  9020 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 4 to 5, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.027796  9264 raft_consensus.cc:2955] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.039227  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet a51a0795b52e4843959cd3be3d077713 with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:44.041370  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet e9a6ffaa328644f4bd7f129f8fa22875 with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:44.059396  9121 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:44.061394  9862 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:44.061848 10427 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:44.066881 10577 tablet_replica.cc:333] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:44.071399 10577 raft_consensus.cc:2243] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:44.077152 10577 raft_consensus.cc:2272] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:44.077363 10578 tablet_replica.cc:333] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:44.078471 10578 raft_consensus.cc:2243] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:44.079150 10578 raft_consensus.cc:2272] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:44.080870 10577 ts_tablet_manager.cc:1916] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:44.082371 10578 ts_tablet_manager.cc:1916] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:44.083389  9120 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:44.085147 10559 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:44.087345  9192 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:44.089854 10427 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:44.089834 10567 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:03:44.109061 10577 ts_tablet_manager.cc:1929] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:03:44.109643 10577 log.cc:1199] T a51a0795b52e4843959cd3be3d077713 P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/a51a0795b52e4843959cd3be3d077713
I20260709 13:03:44.111883  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet a51a0795b52e4843959cd3be3d077713 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:44.156708 10427 raft_consensus.cc:993] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:44.157169 10427 raft_consensus.cc:1081] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:44.160696 10578 ts_tablet_manager.cc:1929] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:03:44.161258  9861 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Advancing to term 2
I20260709 13:03:44.162343  9119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:03:44.162967  9119 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:44.163334  9119 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:44.161295 10578 log.cc:1199] T e9a6ffaa328644f4bd7f129f8fa22875 P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/e9a6ffaa328644f4bd7f129f8fa22875
I20260709 13:03:44.165524  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet e9a6ffaa328644f4bd7f129f8fa22875 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:44.175113  9861 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:44.175414  9119 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:44.176785  9122 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:44.177816  9119 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:44.179597  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:44.180018 10427 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:44.180958 10538 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:44.181316 10538 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:44.181993  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:44.182183 10538 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:44.183259 10538 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:44.186000  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:44.218961  9265 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:03:44.221931  9087 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:44.223098 10558 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:44.223650 10558 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:44.224604 10558 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:44.240232  9018 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.252103  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:44.295709  9121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:44.296255  9121 raft_consensus.cc:606] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:44.298820  9192 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:44.299413  9192 raft_consensus.cc:606] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:44.302006  9192 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:44.302493  9192 raft_consensus.cc:606] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:44.304855  9265 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:44.305384  9265 raft_consensus.cc:606] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:44.306540  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:03:44.308060  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:44.632754 10545 raft_consensus.cc:993] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:44.633239 10518 raft_consensus.cc:1081] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:44.634503  9269 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:44.635161  9269 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:44.635540  9269 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.641094  9269 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:44.642933  9269 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:44.643630  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:44.644316  9121 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.645519  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:44.646004  9192 raft_consensus.cc:3055] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:44.646202  9192 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:44.646835  9192 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:44.647984  9192 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.649503  9121 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:44.650542  9235 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:44.651171 10538 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:44.651649 10538 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:44.652542 10538 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:44.653998  9192 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:44.672513  9020 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.678802  9192 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:44.694469  9269 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:44.695689 10558 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:44.696586 10427 raft_consensus.cc:993] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:44.697039 10538 raft_consensus.cc:1081] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:44.699610  9861 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:03:44.700187  9861 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:44.700493  9861 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:44.707993  9861 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:44.710353  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:44.710897  9193 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:44.711019  9861 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:44.715679 10510 raft_consensus.cc:993] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:44.716557  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:44.717175  9269 raft_consensus.cc:3055] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:44.717446  9269 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:44.718170  9269 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:44.719322  9269 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:44.727205  9193 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:44.728688  9828 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:44.729426 10523 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:44.729910 10523 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:44.730026  9192 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:44.730855 10523 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:44.732421 10589 raft_consensus.cc:1081] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:44.734176 10589 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:44.734606  9192 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38874
I20260709 13:03:44.736307  9192 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:44.737540  9192 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.741381  9861 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Advancing to term 3
I20260709 13:03:44.747215  9192 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:44.750360  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:44.750888  9264 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.765302  9264 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:44.767712  9162 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:44.768592 10585 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:44.769070 10585 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:44.769589  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:44.770169  9121 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:44.769883 10585 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:44.770464  9121 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:44.771152  9269 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:44.771242  9121 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:44.774679  9121 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.773900  9861 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:44.766315  9192 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:44.785717  9121 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:44.791764 10590 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:44.810536 10518 raft_consensus.cc:993] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:44.811044 10567 raft_consensus.cc:1081] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:44.812813  9264 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:44.813339  9264 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:44.813680  9264 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.820600  9264 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:44.834633  9264 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:44.836017  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:44.835866  9018 catalog_manager.cc:5697] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 3 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.836969  9193 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:44.838142  9193 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:44.838972  9193 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:44.839908  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:44.840174  9193 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.842768  9121 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:44.847540 10589 raft_consensus.cc:1064] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:44.848841  9121 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:44.850412  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:44.850324 10589 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:44.851383 10538 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:44.851909 10538 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:44.852687 10538 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:44.858580  9017 catalog_manager.cc:5697] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:44.862946  9861 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:44.863626  9192 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:44.864290 10586 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:44.865065 10588 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:44.878443  9267 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:44.879918 10588 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:44.885166 10586 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.886859  9861 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.893882  9017 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:44.885174  9193 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:44.895774  9192 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.906311  9267 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.909617  9020 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.929659  9121 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:44.934809  9861 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:44.935021  9192 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:44.936367 10586 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:44.937227 10558 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:44.943164 10586 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.944890  9862 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.947631  9193 raft_consensus.cc:2955] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.953487  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9a3eaf34ab444dca94a52a6fce2a556e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:44.964864  9017 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 5 to 6, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:44.984272  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a3eaf34ab444dca94a52a6fce2a556e with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:44.986799 10600 tablet_replica.cc:333] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:44.987792 10600 raft_consensus.cc:2243] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:44.988549 10600 raft_consensus.cc:2272] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:44.992125 10600 ts_tablet_manager.cc:1916] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:45.007702 10600 ts_tablet_manager.cc:1929] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:03:45.008117 10600 log.cc:1199] T 9a3eaf34ab444dca94a52a6fce2a556e P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/9a3eaf34ab444dca94a52a6fce2a556e
I20260709 13:03:45.009675  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 9a3eaf34ab444dca94a52a6fce2a556e (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:45.109566  9122 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:45.115970  9267 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:45.116624  9193 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:45.117834 10590 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:45.118795 10558 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:45.122092  9089 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd9140502bd3469eb208753dea0f3159. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:45.125491  9264 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:45.128336 10544 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:45.134295 10558 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.145637  9268 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.153473  9192 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:45.154443  9069 auto_rebalancer.cc:629] Failed to schedule move for tablet 55b71b09f6834fbbbd500874ff1e14b5: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:03:45.154475  9195 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:45.156741 10593 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:45.158116 10427 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:45.158641  9122 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:45.161368  9267 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:45.165099 10427 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:45.171173  9017 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:45.201808  9195 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.210144  9122 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:45.216832 10599 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:45.219946  9192 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:45.230350  9119 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:45.231887 10585 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:45.245168 10538 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:45.248560  9238 consensus_peers.cc:597] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6cdcabf0b1ec4b319a537ed3662ce253. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:45.248805 10538 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.272524  9267 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:45.274248 10585 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:45.279148  9119 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.292066  9268 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:45.309490  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:45.320648  9195 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.322767  9121 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:45.325556 10427 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:45.326185  9119 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:45.328379  9018 catalog_manager.cc:5697] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.335309 10427 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:45.346350  9195 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:45.349748 10538 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.363605  9119 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.368253 10613 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:03:45.370147  9238 consensus_peers.cc:597] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55cb8917093e4273bb3af2d4d36815cf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:45.385008  9193 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:45.385560  9195 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.398882 10538 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:45.404238  9020 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:45.468307  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:45.524457  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:45.524992  9195 raft_consensus.cc:606] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:45.527813  9268 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:45.528473  9268 raft_consensus.cc:606] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:45.544296  9268 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:45.555236 10617 ts_tablet_manager.cc:933] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:45.557179  9268 raft_consensus.cc:606] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:45.559675  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:03:45.561295  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:45.564779 10617 tablet_copy_client.cc:323] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:45.566181 10538 raft_consensus.cc:993] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:45.566525  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet bd9140502bd3469eb208753dea0f3159 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:45.566749 10613 raft_consensus.cc:1081] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:45.567083  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet bd9140502bd3469eb208753dea0f3159 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-bd9140502bd3469eb208753dea0f3159
I20260709 13:03:45.568490  9193 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38858
I20260709 13:03:45.568990  9193 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:45.569501  9193 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:45.574671  9133 tablet_copy_source_session.cc:215] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:45.578766  9193 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:45.581223 10617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd9140502bd3469eb208753dea0f3159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:45.581570  9193 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:45.581969  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:45.582512  9119 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:45.585726  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:45.586318  9267 raft_consensus.cc:3055] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:45.586721  9267 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:45.587419  9267 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:45.589874  9267 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:45.603087 10617 tablet_copy_client.cc:806] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:45.603725 10585 raft_consensus.cc:993] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:45.603751 10617 tablet_copy_client.cc:670] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:45.604352 10585 raft_consensus.cc:1081] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:45.607103  9119 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:45.608304  9161 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:45.608696  9265 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:45.609246  9265 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:45.609396 10567 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:45.609627  9265 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:45.609906 10567 raft_consensus.cc:697] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:45.610765 10567 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:45.630435  9265 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:45.671828 10617 tablet_copy_client.cc:538] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:45.673506  9265 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:45.674994  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:45.675558  9861 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:45.677701 10620 ts_tablet_manager.cc:933] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:45.656615  9267 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:45.682165 10617 tablet_bootstrap.cc:492] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:45.678414  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:45.637149 10613 raft_consensus.cc:993] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:45.682828 10613 raft_consensus.cc:1081] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:45.684232  9193 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:45.684715  9193 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:45.686673  9193 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:45.688072  9193 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:45.690224  9119 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:03:45.690795  9119 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:45.693922  9119 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:45.694837  9020 catalog_manager.cc:5697] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:45.697055 10620 tablet_copy_client.cc:323] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:45.700140  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 6cdcabf0b1ec4b319a537ed3662ce253 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:45.700697  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 6cdcabf0b1ec4b319a537ed3662ce253 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-6cdcabf0b1ec4b319a537ed3662ce253
I20260709 13:03:45.701834  9861 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:03:45.703150  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:03:45.703979 10538 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:45.704417 10538 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:45.705119 10538 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:45.713967  9119 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:45.716405  9282 tablet_copy_source_session.cc:215] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:45.716665  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:45.717290  9267 raft_consensus.cc:3055] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:45.717603  9267 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:45.718351  9193 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:03:45.718478  9267 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:45.721886  9267 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:45.720782 10620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cdcabf0b1ec4b319a537ed3662ce253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:45.724722 10617 tablet_bootstrap.cc:492] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:45.725675 10617 tablet_bootstrap.cc:492] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:45.726621 10617 ts_tablet_manager.cc:1403] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.045s	user 0.023s	sys 0.009s
I20260709 13:03:45.729610 10617 raft_consensus.cc:359] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:45.730432 10617 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:45.731071 10617 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:45.735141  9020 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 3 to 4, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 4 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:45.739886 10617 ts_tablet_manager.cc:1434] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:03:45.740304  9119 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:45.742981  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:45.743536  9862 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:45.751405  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-bd9140502bd3469eb208753dea0f3159 received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:45.751966  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-bd9140502bd3469eb208753dea0f3159 on tablet bd9140502bd3469eb208753dea0f3159 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:45.754597  9862 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:45.756261  9089 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:45.757644 10589 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:45.758169 10589 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:45.759029 10589 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:45.777423  9267 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:45.787597  9017 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:45.794095 10620 tablet_copy_client.cc:806] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:45.794860 10620 tablet_copy_client.cc:670] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:45.800026 10620 tablet_copy_client.cc:538] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:45.808840 10620 tablet_bootstrap.cc:492] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:45.824056 10617 ts_tablet_manager.cc:933] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:45.832492 10617 tablet_copy_client.cc:323] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:45.834343  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 55cb8917093e4273bb3af2d4d36815cf from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:45.834852  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 55cb8917093e4273bb3af2d4d36815cf from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-55cb8917093e4273bb3af2d4d36815cf
I20260709 13:03:45.840871 10620 tablet_bootstrap.cc:492] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:45.841849 10620 tablet_bootstrap.cc:492] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:45.841944  9282 tablet_copy_source_session.cc:215] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:45.842540 10620 ts_tablet_manager.cc:1403] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.008s
I20260709 13:03:45.844740 10617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55cb8917093e4273bb3af2d4d36815cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:45.845840 10620 raft_consensus.cc:359] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:45.846685 10620 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:45.847285 10620 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:45.849726 10620 ts_tablet_manager.cc:1434] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:45.853714  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-6cdcabf0b1ec4b319a537ed3662ce253 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:45.854141  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-6cdcabf0b1ec4b319a537ed3662ce253 on tablet 6cdcabf0b1ec4b319a537ed3662ce253 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:45.862567 10617 tablet_copy_client.cc:806] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:45.863252 10617 tablet_copy_client.cc:670] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:45.866509 10617 tablet_copy_client.cc:538] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:45.874728 10617 tablet_bootstrap.cc:492] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:45.901532 10617 tablet_bootstrap.cc:492] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:45.903949 10617 tablet_bootstrap.cc:492] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:45.904699 10617 ts_tablet_manager.cc:1403] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.004s
I20260709 13:03:45.907677 10617 raft_consensus.cc:359] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:45.908560 10617 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:45.909190 10617 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:45.910904 10617 ts_tablet_manager.cc:1434] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:03:45.915957  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-55cb8917093e4273bb3af2d4d36815cf received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:45.916443  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-55cb8917093e4273bb3af2d4d36815cf on tablet 55cb8917093e4273bb3af2d4d36815cf with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:46.011365  9267 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:46.012694 10585 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:46.031077  9121 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:46.035535 10518 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:46.099963  9861 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:46.101934 10614 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:46.120543  9862 raft_consensus.cc:1217] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:46.138223  9195 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:03:46.143654  9862 raft_consensus.cc:1217] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:46.153602 10614 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:03:46.167289  9265 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:46.170011 10586 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:03:46.178344  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48124.3us
I20260709 13:03:46.215458  9861 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:46.220517 10558 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:46.357697  9857 raft_consensus.cc:1217] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:03:46.508644 10599 raft_consensus.cc:1064] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:46.511262 10599 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:46.519356  9861 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:46.520763  9195 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:46.523391  9121 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:46.523872 10599 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:46.525467 10538 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:46.527514 10538 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:46.545097 10635 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.549039  9195 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.554076  9857 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.562189  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:46.571337  9017 catalog_manager.cc:5697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.584726  9121 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.642292  9267 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:46.648787  9861 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:46.648882  9195 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:46.650748 10538 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:46.658197 10538 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:46.672820 10614 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.676393  9193 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.680263 10589 raft_consensus.cc:1064] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:46.682817 10589 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:46.683403  9857 raft_consensus.cc:2955] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.701052  9020 catalog_manager.cc:5697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.708647  9268 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:46.708634  9193 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:46.710105 10590 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:46.711606 10558 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:46.715965  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet 6cdcabf0b1ec4b319a537ed3662ce253 with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:46.721184  9857 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:46.722731 10589 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:46.736694  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6cdcabf0b1ec4b319a537ed3662ce253 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:46.742669  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:46.762450 10650 tablet_replica.cc:333] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:46.766136 10650 raft_consensus.cc:2243] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:46.767020 10650 raft_consensus.cc:2272] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:46.772390 10650 ts_tablet_manager.cc:1916] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:46.771677 10558 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.790171 10650 ts_tablet_manager.cc:1929] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:46.790846 10650 log.cc:1199] T 6cdcabf0b1ec4b319a537ed3662ce253 P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/6cdcabf0b1ec4b319a537ed3662ce253
I20260709 13:03:46.791639  9268 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.795997  9861 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.798419  9195 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.792632  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet 6cdcabf0b1ec4b319a537ed3662ce253 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:46.810359  9019 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:46.856417  9121 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:46.860966  9861 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:46.862500  9268 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:46.864550 10558 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:46.865736 10590 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:46.873929  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:46.888389  9195 raft_consensus.cc:606] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:46.876142  9265 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.877776  9861 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.891968  9268 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:46.892491  9268 raft_consensus.cc:606] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:46.882509 10614 raft_consensus.cc:1064] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:46.896764 10614 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:46.897387  9119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:46.873929 10590 raft_consensus.cc:2955] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.914363  9018 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.925819  9005 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bd9140502bd3469eb208753dea0f3159 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:46.926863  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet bd9140502bd3469eb208753dea0f3159 with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:46.927179  9195 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:46.928457  9121 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:46.929031  9119 raft_consensus.cc:606] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:46.929499  9119 raft_consensus.cc:630] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Rejecting request to transfer leadership because tablet server 94504d29c6744f38bb06943479df9466 is not a voter in the active config
I20260709 13:03:46.931407 10614 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:46.934755  9861 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:46.937750  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:03:46.937812 10614 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:46.938828 10634 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:46.939618  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:46.946906 10654 tablet_replica.cc:333] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:46.948098 10654 raft_consensus.cc:2243] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:46.948927 10654 raft_consensus.cc:2272] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:46.960069 10654 ts_tablet_manager.cc:1916] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:46.983819 10538 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:46.990888  9862 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:47.014945  9119 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:47.025681  9018 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:47.035849 10654 ts_tablet_manager.cc:1929] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:47.043959 10654 log.cc:1199] T bd9140502bd3469eb208753dea0f3159 P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/bd9140502bd3469eb208753dea0f3159
I20260709 13:03:47.043144  9192 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:47.046267  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet bd9140502bd3469eb208753dea0f3159 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:47.058125  9268 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:47.067942  9861 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:47.069552 10599 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:47.072094  9119 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:47.077766 10599 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:47.084367 10634 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:47.088459  9121 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:47.088470  9861 raft_consensus.cc:2955] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:47.094961  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 55cb8917093e4273bb3af2d4d36815cf with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:47.105883  9017 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 4 to 5, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:47.118398  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet 55cb8917093e4273bb3af2d4d36815cf with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:47.119634 10654 tablet_replica.cc:333] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:47.120646 10654 raft_consensus.cc:2243] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:47.121281 10654 raft_consensus.cc:2272] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:47.125628 10654 ts_tablet_manager.cc:1916] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:47.138402 10654 ts_tablet_manager.cc:1929] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:03:47.138689 10654 log.cc:1199] T 55cb8917093e4273bb3af2d4d36815cf P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/55cb8917093e4273bb3af2d4d36815cf
I20260709 13:03:47.140115  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet 55cb8917093e4273bb3af2d4d36815cf (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:47.163038 10634 raft_consensus.cc:993] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:47.163586 10599 raft_consensus.cc:1081] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:47.165123  9192 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38858
I20260709 13:03:47.165760  9192 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:47.166118  9192 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:47.172299  9192 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:47.173753  9192 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:47.174813  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:47.174875  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:47.175383  9265 raft_consensus.cc:3055] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:47.175511  9121 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:47.175698  9265 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:47.176601  9265 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:47.177498  9265 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:47.181275  9121 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:47.181655  9265 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:47.182420  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:47.183277 10585 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:47.183719 10585 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:47.184679 10585 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:47.194566  9018 catalog_manager.cc:5697] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:47.351194 10585 raft_consensus.cc:993] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:47.352578 10585 raft_consensus.cc:1081] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:47.354050  9265 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:47.354446  9265 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:47.354661  9265 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:47.359835  9265 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:47.361299  9265 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:47.361899  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:47.362488  9122 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:47.362562  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:47.363174  9192 raft_consensus.cc:3055] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:47.363416  9192 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:47.364189  9192 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:47.365136  9192 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:47.367871  9122 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:47.368891  9235 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:47.369660 10599 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:47.370108 10599 raft_consensus.cc:697] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:47.370543  9192 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:47.370939 10599 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:47.383188  9018 catalog_manager.cc:5697] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 2 to 3, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 3 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:47.566136  9122 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:47.568153 10662 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:47.585201  9265 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:47.586962 10585 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:47.845477  9192 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:47.846995 10599 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:47.861430  9122 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:47.863366 10599 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:47.940572  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:48.115479  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:48.172451  9192 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:48.173032  9192 raft_consensus.cc:606] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:48.175796  9265 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:48.176443  9265 raft_consensus.cc:606] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:48.179610  9119 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:48.180246  9119 raft_consensus.cc:606] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:03:48.181671  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:03:48.183275  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:48.211833  9192 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:48.218511  9164 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55b71b09f6834fbbbd500874ff1e14b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.220346  9268 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:48.220749  9119 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:48.222412 10671 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:48.223243 10662 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:48.233337 10662 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.237044 10648 raft_consensus.cc:993] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:48.237519 10648 raft_consensus.cc:1081] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:03:48.236547  9122 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.239105  9192 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38874
I20260709 13:03:48.239645  9192 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:48.240010  9192 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:48.244349  9265 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.249470  9192 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:48.252019  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:48.254837  9122 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:48.255187  9122 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:48.253608  9268 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:48.255182  9018 catalog_manager.cc:5697] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.253703  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:48.258886  9267 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:48.262712  9122 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:48.263839  9122 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:48.268491  9267 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:48.252259  9192 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:48.269837  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:48.273976 10585 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:48.274477 10585 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:48.275425 10585 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:48.278522  9238 consensus_peers.cc:597] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1abbfa6c01a144acb9bc31470b44be7e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.278918  9192 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.280223  9119 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.281078 10635 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.281936 10599 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.282991  9122 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:48.298048 10635 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.299823 10585 raft_consensus.cc:993] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:48.300314 10662 raft_consensus.cc:1081] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:48.302769  9268 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:48.303364  9268 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:48.303706  9268 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:48.303961  9119 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.301993  9192 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.318905  9268 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:48.324156  9268 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:48.316010  9017 catalog_manager.cc:5697] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.324715  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:48.325259  9121 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:48.323155  9267 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:48.330752  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:48.333587  9192 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:48.336441  9192 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:48.337575  9192 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:48.329393  9020 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.343017  9192 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:48.351230  9121 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:48.372361  9235 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:48.378471 10634 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:48.379101 10634 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:48.380111 10634 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:48.391089  9192 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:48.394322  9195 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:03:48.403571  9238 consensus_peers.cc:597] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc89bbd2f2934bbeaf33619d3d23adfc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.404044  9119 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.405257 10634 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:48.406296 10635 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.416483  9018 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 2 to 3, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 3 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:48.419378 10635 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.424880  9119 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.437729  9195 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.440805  9018 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:48.440449  9192 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:48.454630  9164 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff8d71c72ec64addaf0593eca7fd672d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:48.460475  9265 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.462122  9119 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:03:48.463408 10585 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.474647 10585 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:48.487100 10585 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.495582  9121 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.502759  9267 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.534721  9017 catalog_manager.cc:5697] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:48.647715 10635 raft_consensus.cc:993] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:48.648275 10635 raft_consensus.cc:1081] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:48.649873  9862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:03:48.650382  9862 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:48.650715  9862 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:48.658162  9862 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:48.660660  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:48.661237  9192 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:48.664516  9862 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:48.664695  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:48.665285  9265 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:48.665602  9265 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:48.666291  9265 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:48.667308  9265 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:48.673363  9192 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:03:48.674489  9265 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:03:48.674578  9828 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:48.675529 10660 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:48.676046 10660 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:48.676945 10660 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:48.687368  9018 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 4 to 5, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 5 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:48.770223 10696 ts_tablet_manager.cc:933] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:48.772068 10696 tablet_copy_client.cc:323] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:48.773619  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 1abbfa6c01a144acb9bc31470b44be7e from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:48.774147  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 1abbfa6c01a144acb9bc31470b44be7e from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-1abbfa6c01a144acb9bc31470b44be7e
I20260709 13:03:48.781420 10698 ts_tablet_manager.cc:933] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:48.782117  9282 tablet_copy_source_session.cc:215] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:48.783325 10698 tablet_copy_client.cc:323] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:48.784961  9207 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet 55b71b09f6834fbbbd500874ff1e14b5 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:48.785432 10696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1abbfa6c01a144acb9bc31470b44be7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:48.785511  9207 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet 55b71b09f6834fbbbd500874ff1e14b5 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-55b71b09f6834fbbbd500874ff1e14b5
I20260709 13:03:48.793541  9207 tablet_copy_source_session.cc:215] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:48.798393 10698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b71b09f6834fbbbd500874ff1e14b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:48.805809 10696 tablet_copy_client.cc:806] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:48.806401 10696 tablet_copy_client.cc:670] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:48.810844 10696 tablet_copy_client.cc:538] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:48.817153 10698 tablet_copy_client.cc:806] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:48.817752 10698 tablet_copy_client.cc:670] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:48.821420 10696 tablet_bootstrap.cc:492] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:48.822126 10698 tablet_copy_client.cc:538] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:48.836383 10698 tablet_bootstrap.cc:492] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:48.861991  9121 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:48.868176 10696 tablet_bootstrap.cc:492] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:48.867540 10693 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:48.869527  9192 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:48.872236  9119 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:48.874150 10687 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:48.875957 10687 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:48.909020 10698 tablet_bootstrap.cc:492] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:48.910027 10698 tablet_bootstrap.cc:492] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:48.910763 10698 ts_tablet_manager.cc:1403] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.075s	user 0.026s	sys 0.003s
I20260709 13:03:48.918458 10698 raft_consensus.cc:359] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:48.919421 10698 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:48.920167 10698 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:48.921454  9265 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:48.923182 10585 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:48.930455 10698 ts_tablet_manager.cc:1434] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.004s
I20260709 13:03:48.933125  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-55b71b09f6834fbbbd500874ff1e14b5 received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:48.933670  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-55b71b09f6834fbbbd500874ff1e14b5 on tablet 55b71b09f6834fbbbd500874ff1e14b5 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:48.869524 10696 tablet_bootstrap.cc:492] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:48.956807 10696 ts_tablet_manager.cc:1403] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.136s	user 0.025s	sys 0.014s
I20260709 13:03:48.967718 10696 raft_consensus.cc:359] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:48.972767 10696 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:48.973515 10696 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:48.993675 10696 ts_tablet_manager.cc:1434] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:03:49.006896 10698 ts_tablet_manager.cc:933] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:49.057673  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-1abbfa6c01a144acb9bc31470b44be7e received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:49.058327  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-1abbfa6c01a144acb9bc31470b44be7e on tablet 1abbfa6c01a144acb9bc31470b44be7e with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:49.061362 10711 ts_tablet_manager.cc:933] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:49.078003 10698 tablet_copy_client.cc:323] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:49.083562 10711 tablet_copy_client.cc:323] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:49.085109  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet bc89bbd2f2934bbeaf33619d3d23adfc from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:49.085698  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet bc89bbd2f2934bbeaf33619d3d23adfc from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-bc89bbd2f2934bbeaf33619d3d23adfc
I20260709 13:03:49.092947  9282 tablet_copy_source_session.cc:215] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:49.102797 10698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc89bbd2f2934bbeaf33619d3d23adfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:49.104136  9207 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet ff8d71c72ec64addaf0593eca7fd672d from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:49.104719  9207 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet ff8d71c72ec64addaf0593eca7fd672d from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-ff8d71c72ec64addaf0593eca7fd672d
I20260709 13:03:49.119254  9207 tablet_copy_source_session.cc:215] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:49.123621 10711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff8d71c72ec64addaf0593eca7fd672d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:49.129979 10698 tablet_copy_client.cc:806] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:49.130645 10698 tablet_copy_client.cc:670] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:49.133755 10698 tablet_copy_client.cc:538] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:49.140165 10698 tablet_bootstrap.cc:492] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:49.140583 10711 tablet_copy_client.cc:806] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:49.141170 10711 tablet_copy_client.cc:670] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:49.146127 10711 tablet_copy_client.cc:538] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:49.150307  9192 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:03:49.151787 10659 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:49.161496 10711 tablet_bootstrap.cc:492] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:49.190343  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:49.216811  9861 raft_consensus.cc:1217] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:03:49.244390 10711 tablet_bootstrap.cc:492] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:49.245738 10711 tablet_bootstrap.cc:492] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:49.246740 10711 ts_tablet_manager.cc:1403] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.086s	user 0.025s	sys 0.011s
I20260709 13:03:49.250523 10711 raft_consensus.cc:359] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:49.251770 10711 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:49.252524 10711 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:49.260459 10711 ts_tablet_manager.cc:1434] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:03:49.264246  9265 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:03:49.265460 10660 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:03:49.308511  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-ff8d71c72ec64addaf0593eca7fd672d received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:49.309968  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-ff8d71c72ec64addaf0593eca7fd672d on tablet ff8d71c72ec64addaf0593eca7fd672d with peer 92919b70789041c8a0de0b3525154fd1
W20260709 13:03:49.325969  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 58244.9us
I20260709 13:03:49.331911 10698 tablet_bootstrap.cc:492] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:49.342160 10698 tablet_bootstrap.cc:492] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:49.344344 10698 ts_tablet_manager.cc:1403] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.204s	user 0.046s	sys 0.063s
I20260709 13:03:49.478317 10698 raft_consensus.cc:359] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:49.481544 10698 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:49.482374 10698 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:49.510854 10698 ts_tablet_manager.cc:1434] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.164s	user 0.004s	sys 0.008s
I20260709 13:03:49.599731  9282 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-bc89bbd2f2934bbeaf33619d3d23adfc received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:49.603698  9282 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-bc89bbd2f2934bbeaf33619d3d23adfc on tablet bc89bbd2f2934bbeaf33619d3d23adfc with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:49.622440  9860 raft_consensus.cc:1217] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.626755  9861 raft_consensus.cc:1217] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.633538  9862 raft_consensus.cc:1217] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:03:49.826023 10585 raft_consensus.cc:1064] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:49.828852 10585 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:49.838006  9265 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:49.838392  9860 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:49.838052  9122 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:49.840544 10693 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:49.841768 10585 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:49.842706 10730 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:49.871762 10693 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:49.873623  9122 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:49.888381  9860 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:49.894155  9265 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:49.898429  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:49.912941  9018 catalog_manager.cc:5697] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:49.946089  9196 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:49.955353  9860 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:49.957108 10585 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:49.959560  9122 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:49.964188 10730 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:49.968089 10693 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:49.988651  9860 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:49.993762  9265 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:49.994663  9265 raft_consensus.cc:606] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:50.008095  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 55b71b09f6834fbbbd500874ff1e14b5 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:50.013063  9020 catalog_manager.cc:5697] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 4 to 5, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:50.015969  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:03:50.016673  9195 raft_consensus.cc:606] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:50.020581  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:03:50.022454  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:50.023530  9122 raft_consensus.cc:2955] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:50.028422 10725 raft_consensus.cc:1064] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
W20260709 13:03:50.028977 10725 raft_consensus.cc:1068] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Unable to promote non-voter 92919b70789041c8a0de0b3525154fd1: Service unavailable: leader transfer in progress
I20260709 13:03:50.047917  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 55b71b09f6834fbbbd500874ff1e14b5 with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:50.050616 10740 tablet_replica.cc:333] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:50.052918 10740 raft_consensus.cc:2243] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:50.054178 10740 raft_consensus.cc:2272] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:50.058584 10740 ts_tablet_manager.cc:1916] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:50.076938 10740 ts_tablet_manager.cc:1929] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:03:50.077518 10740 log.cc:1199] T 55b71b09f6834fbbbd500874ff1e14b5 P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/55b71b09f6834fbbbd500874ff1e14b5
I20260709 13:03:50.083840  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 55b71b09f6834fbbbd500874ff1e14b5 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:50.116173 10693 raft_consensus.cc:1064] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:50.119175 10693 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:50.146312 10585 raft_consensus.cc:993] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:50.146840 10730 raft_consensus.cc:1081] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:50.149055  9267 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:03:50.149755  9267 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:50.150214  9267 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.157976 10724 raft_consensus.cc:1064] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:50.165650 10724 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:50.176563  9267 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:50.179967  9267 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:50.194161  9857 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.220239 10729 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:50.226876  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:50.227226  9121 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.227631  9122 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.245225  9122 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:50.269935  9122 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:50.414017  9235 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:50.417949 10725 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:50.418462 10723 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:50.419032 10723 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:50.431630  9861 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.432435  9270 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:03:50.433534  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 213818us
I20260709 13:03:50.448714 10585 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:50.454416 10721 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:50.419906 10723 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:50.475593 10635 raft_consensus.cc:993] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:50.476070 10635 raft_consensus.cc:1081] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
W20260709 13:03:50.567833  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 60924us
I20260709 13:03:50.582801 10585 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:50.583134  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:50.583716  9191 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:50.584062  9191 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:50.613495  9191 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:50.619524  9121 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:03:50.620100  9121 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:50.620438  9121 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.619271 10739 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:50.623607  9190 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.644654  9191 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:50.659322  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 87408.7us
W20260709 13:03:50.700706 10748 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:03:50.695619  9121 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:50.657455 10723 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:03:50.754743  9121 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:50.796782  9860 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:50.798003  9191 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
W20260709 13:03:50.803491  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 161018us
W20260709 13:03:50.800588 10585 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:03:50.817188  9019 catalog_manager.cc:5697] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:50.837378  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 82859.9us
I20260709 13:03:50.839658  9268 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:50.842083  9017 catalog_manager.cc:5697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:50.851310  9115 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:03:50.855417  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:50.856010  9192 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.860806 10635 raft_consensus.cc:1064] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
W20260709 13:03:50.861632 10635 raft_consensus.cc:1068] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Unable to promote non-voter 92919b70789041c8a0de0b3525154fd1: Service unavailable: leader transfer in progress
I20260709 13:03:50.863205  9192 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:50.864924 10635 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:50.868531  9089 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:50.869562 10758 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:50.870126 10758 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:50.871023 10758 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:50.873709  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:50.874286  9270 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:50.874574  9270 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:50.875241  9270 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:50.876782  9270 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:50.884774  9270 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:50.902913 10761 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:50.912748  9115 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:50.928658  9018 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:50.954335  9120 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:50.965042  9860 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:50.970589  9192 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.025219  9017 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.030678  9121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:51.031422  9121 raft_consensus.cc:606] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:03:51.036083  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:51.036193  9121 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:51.036943  9121 raft_consensus.cc:606] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:03:51.038082  9005 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 1abbfa6c01a144acb9bc31470b44be7e already in progress [suppressed 1 similar messages]
I20260709 13:03:51.043658  9120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:51.051011  9120 raft_consensus.cc:606] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:03:51.080410  9069 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:03:51.080856  9069 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 1abbfa6c01a144acb9bc31470b44be7e already in progress
W20260709 13:03:51.081149  9069 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 1abbfa6c01a144acb9bc31470b44be7e already in progress
I20260709 13:03:51.083065 10765 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:51.105671  9120 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:51.105057  9191 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.107285 10765 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:51.137135  9270 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:51.146399  9120 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.148423  9860 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.148358 10771 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:51.150555 10635 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:51.157891 10635 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.159199  9115 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.177217  9860 raft_consensus.cc:2955] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.208108  9018 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 3 to 4, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 1 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.218837  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bc89bbd2f2934bbeaf33619d3d23adfc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:51.231025  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet bc89bbd2f2934bbeaf33619d3d23adfc with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38856
W20260709 13:03:51.256803  9144 debug-util.cc:398] Leaking SignalData structure 0x7b08000ca240 after lost signal to thread 8024
W20260709 13:03:51.258066  9144 debug-util.cc:398] Leaking SignalData structure 0x7b080018fb00 after lost signal to thread 9055
W20260709 13:03:51.259366  9144 debug-util.cc:398] Leaking SignalData structure 0x7b080035af20 after lost signal to thread 9147
W20260709 13:03:51.261202  9144 debug-util.cc:398] Leaking SignalData structure 0x7b080037f420 after lost signal to thread 9222
W20260709 13:03:51.263465  9144 debug-util.cc:398] Leaking SignalData structure 0x7b0800376980 after lost signal to thread 9296
W20260709 13:03:51.264665  9144 debug-util.cc:398] Leaking SignalData structure 0x7b08002dbb80 after lost signal to thread 9887
I20260709 13:03:51.270951 10781 tablet_replica.cc:333] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:03:51.279770 10781 raft_consensus.cc:2243] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:51.281021 10781 raft_consensus.cc:2272] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:51.284889 10781 ts_tablet_manager.cc:1916] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:51.306250 10781 ts_tablet_manager.cc:1929] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:51.306746 10781 log.cc:1199] T bc89bbd2f2934bbeaf33619d3d23adfc P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/bc89bbd2f2934bbeaf33619d3d23adfc
I20260709 13:03:51.308660  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet bc89bbd2f2934bbeaf33619d3d23adfc (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:51.330070  9192 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:51.336187  9860 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.337661 10739 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:51.348054 10745 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.357218  9860 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.370520  9115 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:03:51.372740  9269 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:51.372845 10763 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:51.375895 10743 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:51.405486  9860 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 1 LEARNER]: Advancing to term 2
I20260709 13:03:51.386510  9121 raft_consensus.cc:2955] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.437160  9191 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:51.433277  9019 catalog_manager.cc:5697] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 1 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:51.447403 10758 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:51.460996  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet ff8d71c72ec64addaf0593eca7fd672d with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39750
I20260709 13:03:51.449375  9860 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:51.487149 10743 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:51.497355 10784 tablet_replica.cc:333] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:03:51.500020  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ff8d71c72ec64addaf0593eca7fd672d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:51.500196 10784 raft_consensus.cc:2243] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:51.501228 10784 raft_consensus.cc:2272] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:51.505424 10784 ts_tablet_manager.cc:1916] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:51.540091 10784 ts_tablet_manager.cc:1929] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:03:51.543942 10784 log.cc:1199] T ff8d71c72ec64addaf0593eca7fd672d P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/ff8d71c72ec64addaf0593eca7fd672d
I20260709 13:03:51.546556  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet ff8d71c72ec64addaf0593eca7fd672d (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:51.594018 10743 raft_consensus.cc:993] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:51.594576 10743 raft_consensus.cc:1081] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:51.597560  9264 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39754
I20260709 13:03:51.598116  9264 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:51.598512  9264 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:51.618045  9264 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:51.621673 10783 raft_consensus.cc:1064] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:51.623642  9264 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:51.625662  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:51.626618  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:51.627173  9192 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:51.629940  9121 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:51.630328  9121 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:03:51.630535 10783 raft_consensus.cc:1068] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: Unable to promote non-voter 92919b70789041c8a0de0b3525154fd1: Service unavailable: leader transfer in progress
I20260709 13:03:51.630997  9121 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:51.632583  9121 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:51.634790  9192 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:51.640972  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:51.642102 10725 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:51.642586 10725 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:51.643461 10725 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:51.654484  9121 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:51.666286  9019 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 2 to 3, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 3 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:51.851312  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:51.901240  9264 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:51.901787  9264 raft_consensus.cc:606] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:51.904337  9264 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:03:51.904876  9264 raft_consensus.cc:606] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:51.906020  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:03:51.907238  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:51.966600 10725 raft_consensus.cc:993] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:51.967208 10771 raft_consensus.cc:1081] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:03:51.970146  9120 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:03:51.970793  9120 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:51.971067  9120 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:51.976006  9120 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:51.978696  9120 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:51.979871  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:51.980517  9264 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:51.980335  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:51.980798  9264 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:51.981076  9194 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:51.981539  9264 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:51.982489  9264 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:51.989552  9264 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:03:51.991134  9087 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:51.992697  9194 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:03:51.992949 10783 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:51.993521 10783 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:51.994517 10783 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:52.011004  9019 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 4, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 4 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:52.028003 10635 raft_consensus.cc:993] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:52.028620 10635 raft_consensus.cc:1081] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:03:52.030088  9860 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:03:52.030527  9860 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:52.030795  9860 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:52.038335  9860 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:52.045747  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:52.046485  9120 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:52.046904  9860 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:52.049079  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:52.049714  9264 raft_consensus.cc:3055] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:52.050017  9264 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:52.050927  9264 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:52.052264  9264 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:52.054648  9120 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:52.056031  9825 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:52.056936 10759 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:52.057451 10759 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:52.058463 10759 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:52.059826  9264 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:52.070041  9019 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 3 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:52.210590  9860 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Advancing to term 3
I20260709 13:03:52.218119  9860 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:52.219960 10725 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:52.230774  9194 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:52.235092 10771 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:52.236065 10725 raft_consensus.cc:1064] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
W20260709 13:03:52.236552 10725 raft_consensus.cc:1068] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Unable to promote non-voter 92919b70789041c8a0de0b3525154fd1: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:03:52.253115  9120 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:52.255640 10725 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:03:52.305639 10635 raft_consensus.cc:1064] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:52.308760 10635 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:52.316396  9860 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:52.317067  9120 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:52.317724  9194 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:03:52.318213 10635 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:52.320025 10725 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:52.321681 10725 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:52.329037 10635 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:52.333214  9860 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:52.337399  9194 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:52.350605  9020 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:52.360687  9120 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:52.374676  9264 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:52.386772  9859 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:52.387874  9194 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:52.388805 10795 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:52.389750 10796 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:52.400357 10796 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:52.413720  9860 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:52.414529  9192 raft_consensus.cc:2955] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:52.439200  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1abbfa6c01a144acb9bc31470b44be7e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:03:52.442662  9019 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 5 to 6, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 3 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:52.462864  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet 1abbfa6c01a144acb9bc31470b44be7e with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43778
I20260709 13:03:52.478646  9268 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:52.482401 10762 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:03:52.492358 10800 tablet_replica.cc:333] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:03:52.518335  9120 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:52.517899  9264 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:52.518906 10800 raft_consensus.cc:2243] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:52.520959 10789 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:52.524689 10762 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:52.527364 10800 raft_consensus.cc:2272] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:52.552476  9194 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:03:52.557580 10788 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:52.562247 10800 ts_tablet_manager.cc:1916] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:03:52.608781 10800 ts_tablet_manager.cc:1929] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:03:52.609167 10800 log.cc:1199] T 1abbfa6c01a144acb9bc31470b44be7e P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/1abbfa6c01a144acb9bc31470b44be7e
I20260709 13:03:52.612876  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet 1abbfa6c01a144acb9bc31470b44be7e (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:03:52.638052  9194 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:52.644687  9269 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:52.645771  9164 consensus_peers.cc:597] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98d913b72e5b43e8b8f20fb99d0838c4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.647008  9121 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:52.647334 10763 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.648478 10761 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:52.669060 10745 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.673588  9120 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.679687  9269 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.694003  9020 catalog_manager.cc:5697] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 2, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.716224  9264 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:52.753278  9238 consensus_peers.cc:597] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 991b28944986479a8ad096ca92f1a435. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.757887  9194 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:52.760119  9120 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:52.762387 10635 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:52.763541 10796 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:52.773602 10795 raft_consensus.cc:2955] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.776007  9120 raft_consensus.cc:2955] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.785321  9192 raft_consensus.cc:2955] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.797272  9020 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.814262  9120 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:52.821822  9269 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:03:52.823235 10789 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:52.825713  9192 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:03:52.827857  9089 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 017b65b967c44e7688e66e7aab235b62. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.828742 10783 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:52.833114 10783 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.836544  9269 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.860064  9269 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:52.859827  9194 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.863054  9019 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 5, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 4 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.867470  9192 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:03:52.867477  9238 consensus_peers.cc:597] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a4bd074bf594b89b28b47d7482e8ac1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:52.870097 10796 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:52.871044  9120 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:52.872506 10796 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:52.899243 10796 raft_consensus.cc:2955] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.904410  9120 raft_consensus.cc:2955] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.908131  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:52.923467  9192 raft_consensus.cc:2955] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:52.945964  9020 catalog_manager.cc:5697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 3, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:03:53.137077 10818 ts_tablet_manager.cc:933] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:53.152767 10818 tablet_copy_client.cc:323] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:03:53.154472  9207 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet 98d913b72e5b43e8b8f20fb99d0838c4 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:03:53.155157  9207 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet 98d913b72e5b43e8b8f20fb99d0838c4 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-98d913b72e5b43e8b8f20fb99d0838c4
I20260709 13:03:53.165233  9207 tablet_copy_source_session.cc:215] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:53.171974 10818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d913b72e5b43e8b8f20fb99d0838c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.190630 10818 tablet_copy_client.cc:806] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:53.191226 10818 tablet_copy_client.cc:670] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:53.200284 10818 tablet_copy_client.cc:538] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:53.208734 10821 ts_tablet_manager.cc:933] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:53.213013 10818 tablet_bootstrap.cc:492] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:53.231933 10821 tablet_copy_client.cc:323] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.65:39939
I20260709 13:03:53.237406  9133 tablet_copy_service.cc:140] P ce51b61da22f40d4aa0204a6b1028f28: Received BeginTabletCopySession request for tablet 017b65b967c44e7688e66e7aab235b62 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:47150)
I20260709 13:03:53.237968  9133 tablet_copy_service.cc:161] P ce51b61da22f40d4aa0204a6b1028f28: Beginning new tablet copy session on tablet 017b65b967c44e7688e66e7aab235b62 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:47150: session id = 92919b70789041c8a0de0b3525154fd1-017b65b967c44e7688e66e7aab235b62
I20260709 13:03:53.249015  9133 tablet_copy_source_session.cc:215] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:53.257714 10821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 017b65b967c44e7688e66e7aab235b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.267057 10818 tablet_bootstrap.cc:492] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.268473 10818 tablet_bootstrap.cc:492] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:53.269239 10818 ts_tablet_manager.cc:1403] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.057s	user 0.021s	sys 0.019s
I20260709 13:03:53.276724 10821 tablet_copy_client.cc:806] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:53.277427 10821 tablet_copy_client.cc:670] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:53.277179 10818 raft_consensus.cc:359] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:53.278191 10818 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:53.279006 10818 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:53.282367 10818 ts_tablet_manager.cc:1434] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:03:53.282713 10821 tablet_copy_client.cc:538] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:53.286201  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-98d913b72e5b43e8b8f20fb99d0838c4 received from {username='slave'} at 127.0.0.1:56646
I20260709 13:03:53.286731  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-98d913b72e5b43e8b8f20fb99d0838c4 on tablet 98d913b72e5b43e8b8f20fb99d0838c4 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:53.321180 10821 tablet_bootstrap.cc:492] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:53.335695 10824 ts_tablet_manager.cc:933] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:53.350268 10818 ts_tablet_manager.cc:933] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:53.356561 10824 tablet_copy_client.cc:323] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:53.366499  9282 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 8a4bd074bf594b89b28b47d7482e8ac1 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:53.367035  9282 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 8a4bd074bf594b89b28b47d7482e8ac1 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-8a4bd074bf594b89b28b47d7482e8ac1
I20260709 13:03:53.369901 10818 tablet_copy_client.cc:323] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:03:53.377943  9282 tablet_copy_source_session.cc:215] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:53.378875  9281 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 991b28944986479a8ad096ca92f1a435 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:03:53.379416  9281 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 991b28944986479a8ad096ca92f1a435 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-991b28944986479a8ad096ca92f1a435
I20260709 13:03:53.386567  9281 tablet_copy_source_session.cc:215] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:53.389403 10824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4bd074bf594b89b28b47d7482e8ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.390290 10818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 991b28944986479a8ad096ca92f1a435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.426978 10821 tablet_bootstrap.cc:492] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.429865 10821 tablet_bootstrap.cc:492] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:53.430601 10821 ts_tablet_manager.cc:1403] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.110s	user 0.027s	sys 0.006s
I20260709 13:03:53.433789 10821 raft_consensus.cc:359] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:53.437062 10824 tablet_copy_client.cc:806] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:53.437933 10824 tablet_copy_client.cc:670] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:53.442706 10824 tablet_copy_client.cc:538] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:53.449132 10818 tablet_copy_client.cc:806] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:53.449770 10818 tablet_copy_client.cc:670] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:53.450217 10821 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:53.454968 10821 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:53.465898 10818 tablet_copy_client.cc:538] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:53.473688 10821 ts_tablet_manager.cc:1434] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:03:53.476589 10824 tablet_bootstrap.cc:492] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:53.477525 10818 tablet_bootstrap.cc:492] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:03:53.478289  9133 tablet_copy_service.cc:342] P ce51b61da22f40d4aa0204a6b1028f28: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-017b65b967c44e7688e66e7aab235b62 received from {username='slave'} at 127.0.0.1:47150
I20260709 13:03:53.478765  9133 tablet_copy_service.cc:434] P ce51b61da22f40d4aa0204a6b1028f28: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-017b65b967c44e7688e66e7aab235b62 on tablet 017b65b967c44e7688e66e7aab235b62 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:53.520964 10824 tablet_bootstrap.cc:492] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.521893 10824 tablet_bootstrap.cc:492] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:53.522565 10824 ts_tablet_manager.cc:1403] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.046s	user 0.027s	sys 0.003s
I20260709 13:03:53.525727 10824 raft_consensus.cc:359] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:53.526867 10824 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:53.527575 10824 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:53.531896 10824 ts_tablet_manager.cc:1434] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.004s
I20260709 13:03:53.539359  9281 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-8a4bd074bf594b89b28b47d7482e8ac1 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:53.539870  9281 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-8a4bd074bf594b89b28b47d7482e8ac1 on tablet 8a4bd074bf594b89b28b47d7482e8ac1 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:53.551578 10818 tablet_bootstrap.cc:492] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:53.552649 10818 tablet_bootstrap.cc:492] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:03:53.553308 10818 ts_tablet_manager.cc:1403] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 0.076s	user 0.023s	sys 0.005s
I20260709 13:03:53.556356 10818 raft_consensus.cc:359] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:53.557843 10818 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:03:53.559665 10818 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:53.565776 10818 ts_tablet_manager.cc:1434] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.003s
I20260709 13:03:53.568434  9281 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-991b28944986479a8ad096ca92f1a435 received from {username='slave'} at 127.0.0.1:38650
I20260709 13:03:53.568863  9281 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-991b28944986479a8ad096ca92f1a435 on tablet 991b28944986479a8ad096ca92f1a435 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:54.035346  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:03:54.867834  9089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.000s	sys 0.002s
W20260709 13:03:54.868490  9089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.000s	sys 0.003s
W20260709 13:03:54.880007  9828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.001s	sys 0.000s
W20260709 13:03:54.880589  9828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.002s	sys 0.000s
I20260709 13:03:54.928767 10838 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:54.930850 10835 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:54.930123 10832 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:54.931432 10835 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.879557 10837 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:54.932574 10837 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:54.931552 10832 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.934592 10832 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:54.930810 10838 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.936355 10833 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:54.937148 10843 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:54.936918 10833 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.934597 10837 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:54.943010 10833 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:54.938338 10838 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:54.938680 10834 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:54.951380 10834 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.938246 10844 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:54.954411 10838 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:54.955027 10838 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.957612 10842 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:54.958549 10842 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.960747 10842 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:54.961282 10842 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:54.961787 10842 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.964099 10842 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:54.947005 10837 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:54.974291 10837 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.976713 10837 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:54.954141 10844 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.939153 10843 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.964767 10831 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:54.978780 10844 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:54.965250 10832 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:54.983289 10832 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:54.985801 10832 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:54.978730 10831 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.986577 10840 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:54.987196 10840 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.987643  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:54.967428 10838 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:54.968268  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:54.988438  9853 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:54.988814  9862 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:54.989117 10831 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:54.993005 10842 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:54.993302  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:54.993566 10842 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:54.997171 10832 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:54.997735 10832 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:54.998953 10837 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:54.999469 10837 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:54.999678 10832 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:54.981501 10854 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:54.947716 10835 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.001060 10854 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:55.001575 10837 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:54.990272 10840 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.004144 10840 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.004539 10852 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.004871 10840 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:55.012224 10836 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.015367 10838 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.019184 10836 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.019289 10838 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.021636 10836 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.005124 10852 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.022109 10838 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.042048 10841 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.042471  9855 raft_consensus.cc:1217] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.4->[4.5-4.5]   Dedup: 4.5->[]
I20260709 13:03:55.042763 10841 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.044545  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.024813 10852 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.025236 10843 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.010785 10835 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.056579 10852 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.056811 10835 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:03:55.057706  9088 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.004s	sys 0.000s
I20260709 13:03:55.057309 10852 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:55.058063  9088 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.004s	sys 0.000s
I20260709 13:03:55.059087 10835 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.010535 10848 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.027263 10840 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.027665 10847 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.060334 10847 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.062630 10847 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.011580 10839 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.028483 10846 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.063347 10839 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.063447 10846 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.065629 10839 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.029443 10851 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.066076 10846 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.066340 10851 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.030225  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.030174  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.003507 10834 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.011106 10850 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.068996 10850 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.069022 10848 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.014871 10842 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.071921 10848 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.031431 10854 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.017758  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:03:55.072934  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55289.1us
I20260709 13:03:55.073674  9089 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:55.075235 10831 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.075769 10831 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:55.078089 10831 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.078394 10789 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:55.078779 10789 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.079123 10789 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:55.081405 10832 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.081941 10832 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.071617 10850 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.083254 10850 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.077975 10837 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:55.085181  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.088256  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.089010  9862 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:55.090783 10840 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.091383 10840 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.083850 10850 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.093569 10850 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.094251 10840 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.095191 10833 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.095758 10851 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.097431  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.097914 10841 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.098331  9860 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:55.098881 10837 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.099324 10837 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:55.095784 10833 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.101675 10833 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.102679 10832 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.107941 10832 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.112025 10832 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.113441 10833 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.114413 10839 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.114264 10833 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.114264  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:55.117070 10833 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.115039 10839 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.114351 10832 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.034003 10845 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.034736 10856 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.036617 10858 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.022725  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:55.123239 10839 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.137336  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:55.159888  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.033628 10860 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.165907  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:55.167695  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.173393  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:55.182588  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.184249  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:55.174090  9196 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:55.123911 10845 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.186592  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.187489  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:55.191591  9196 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:55.156746 10788 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.167160 10840 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.166898 10856 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.180467 10860 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:55.224540 10856 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.167022 10858 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.227269 10858 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.230744 10859 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.231313 10859 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.193466 10832 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.193645 10833 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.243419 10845 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.195883 10839 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.195699 10831 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:03:55.263885  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 100375us
I20260709 13:03:55.271951  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.220391 10788 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.219215 10857 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.302937 10845 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.297602  9119 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:55.301529  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:55.301723  9052 debug-util.cc:398] Leaking SignalData structure 0x7b0800091ae0 after lost signal to thread 8024
I20260709 13:03:55.301808 10855 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.272197 10842 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.294799  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.221630 10840 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.314206 10840 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:03:55.314320  9052 debug-util.cc:398] Leaking SignalData structure 0x7b080008a1c0 after lost signal to thread 9055
I20260709 13:03:55.310146 10852 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.308282 10862 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.308290 10867 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.245251 10836 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.241326 10858 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.320238 10862 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.303053 10857 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.303519 10845 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.322167 10788 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.322748  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.279021 10833 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.278743 10841 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:03:55.323783  9052 debug-util.cc:398] Leaking SignalData structure 0x7b080018e540 after lost signal to thread 9147
I20260709 13:03:55.321691 10842 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:55.348978  9052 debug-util.cc:398] Leaking SignalData structure 0x7b080022bce0 after lost signal to thread 9222
I20260709 13:03:55.349400 10842 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.305236 10839 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:03:55.350526  9052 debug-util.cc:398] Leaking SignalData structure 0x7b0800318940 after lost signal to thread 9296
I20260709 13:03:55.351416 10839 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.309227 10860 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.352329 10859 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.328009 10836 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.347114 10871 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.297053 10831 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.344720 10867 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.252592 10832 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.343255 10788 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.370975 10857 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.372651 10862 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.373813  9115 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:55.336436 10858 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.367528 10788 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:03:55.382481  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 83873.1us
W20260709 13:03:55.409657  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 63315.7us
W20260709 13:03:55.369815  9052 debug-util.cc:398] Leaking SignalData structure 0x7b0800359ca0 after lost signal to thread 9887
I20260709 13:03:55.373550 10856 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.321988 10855 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.370157 10840 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.354058 10841 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.416105 10872 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.416677 10874 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.374078 10842 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.409161 10860 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.424252 10860 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.426365 10860 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.381963 10854 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.419167 10856 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.421658 10862 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.427414 10854 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.427852 10870 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.427558 10862 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.421729 10853 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.422648 10858 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.379930 10866 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.430560  9161 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:55.380051 10863 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.432837 10856 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.428404 10870 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.435961 10865 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.436581 10862 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.367305 10871 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.438465  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:55.442221  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:55.429975 10840 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.440177 10851 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:55.481369 10858 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.441682 10850 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.437341 10842 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.489660 10864 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.438651 10872 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.492125  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:55.485478  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 47118.6us
I20260709 13:03:55.491609 10858 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:55.493521 10840 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.496531 10854 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.497133 10875 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.497537 10858 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.499480  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.500108  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.488673 10851 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.513005 10851 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:55.437834 10853 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.511696 10831 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.491942 10850 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.510895  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.494089 10862 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.483623 10837 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.516851  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.527856  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.513952 10789 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.498409 10852 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:55.508706 10838 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:55.436748 10874 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.501132 10845 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.490208 10864 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.540102 10842 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.501523 10867 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.514667 10833 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.497768 10875 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.536230 10836 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.545707  9120 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:55.488265 10863 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.541452 10838 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.537735 10852 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.543835 10832 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.486408 10866 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.552500  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.551966 10841 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.549800 10869 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.548784 10788 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.538453 10862 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.550056 10855 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.550446 10845 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.501833 10865 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.554497 10838 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:55.554394 10852 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:55.555727 10858 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.560132 10870 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.566696 10876 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.574856  9860 raft_consensus.cc:1217] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:03:55.575575  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.577355 10850 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.570899 10858 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.578204 10851 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:55.580355 10858 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.581804 10864 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.585742  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.587106  9123 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:55.589148  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.589879  9121 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:55.590874  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.593174 10863 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.593674  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.596632  9860 raft_consensus.cc:1217] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:03:55.598248  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.600765 10876 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.602068  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.607488  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.610260  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.640323  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.636029  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.640977  9857 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:55.643982  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.644563 10853 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:03:55.651062  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 208915us
I20260709 13:03:55.642037  9123 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:55.571045 10845 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.570278 10869 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:55.653525  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:55.696527  9857 raft_consensus.cc:1217] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:03:55.645429 10871 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.684063  9117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:55.708819  9117 raft_consensus.cc:606] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:55.678169 10861 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:03:55.710403 10861 raft_consensus.cc:3055] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:55.710798 10861 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:55.712767 10856 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:55.712911 10861 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.713361 10856 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:03:55.712057  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 58608.3us
I20260709 13:03:55.711333 10874 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:03:55.689385  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 145942us
I20260709 13:03:55.713995 10856 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:55.742795  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:03:55.762200 10837 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.837318  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:55.903258 10863 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.903401  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.903635 10879 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.904234 10879 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.904605  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.903788 10863 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.887274 10861 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:55.888080  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 173574us
I20260709 13:03:55.915735  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.916285  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.916580  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.918272  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.915923  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.888541 10789 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.890712 10869 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.919054  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:55.971400  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.890156 10852 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.891734  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.892207 10851 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.892879  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.893276 10865 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.893719 10864 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:55.981290  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.981251 10864 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.982143  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.983804 10864 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.984207  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.895083 10876 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:03:55.895551  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 90493.3us
I20260709 13:03:55.896189 10872 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.684007 10853 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:55.899412 10875 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.900317 10862 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:55.901630 10845 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.991062  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:55.901800 10874 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.902199 10850 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.991853 10874 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:55.991927 10850 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.902264 10858 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.993813 10874 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:55.994062 10858 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.994562  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:55.996133 10858 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.902638  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:55.902735 10860 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.997129 10872 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:55.997365 10860 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:55.998777 10854 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:55.999123 10854 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:55.908253 10863 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:55.909813 10868 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:55.977108 10865 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:55.990346  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.012215  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:55.996003  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.004088  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.016642  9270 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:56.004617  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.005065  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.005920  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.017678  9194 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:56.008311  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.019519  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.019940  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.009438  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.021389  9268 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:03:55.990201 10853 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.020431  9270 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:56.025036  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.018528  9193 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:56.022692  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.023072  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:55.999421 10860 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.026746  9192 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:56.030812  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:55.993827 10850 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.025528  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.051396  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.047323  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.052353  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.054349  9194 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:56.053275  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.058178  9265 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:56.062651  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.063383  9194 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:56.068622  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.069252  9193 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:56.069373  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:56.070698  9194 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
W20260709 13:03:56.076786  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 164334us
I20260709 13:03:56.078133  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:56.081142  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.046502  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.107399  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.032262 10866 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:56.018929 10868 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.144783  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.051505 10879 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:56.011107 10856 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.020680 10865 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.167483  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.183378  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.164911  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:56.182330  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.008556 10838 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.068166 10861 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:56.166087  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.182013  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.167541  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.175000  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.169041  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.212966  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.209892  9271 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:03:56.213593  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.161233  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:56.217658  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.219380  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.107411 10860 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:56.220799  9270 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:56.217765  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.219380  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:56.217947  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 160221us
I20260709 13:03:56.165237 10868 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.222239  9269 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:56.222675  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:56.222759  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.166591  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.223335  9114 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:56.223909  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.162212  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.225838  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.160480  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.167912  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.227483  9189 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:03:56.230480  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.228859  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:56.236900  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:56.220793 10860 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:56.269613  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.279122 10838 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.280861 10856 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.281319 10884 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:56.281843 10884 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:56.282298 10884 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:56.230013 10836 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:56.309514 10836 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:56.310045 10836 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:56.267642  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.313247 10852 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:56.268064 10860 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:56.228859  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:03:56.294625  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 57337.8us
I20260709 13:03:56.306354  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.337827 10865 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.307797  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:56.307999  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.218606 10861 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.381734  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.383651  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.387701  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.388273  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.389357  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.390900  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.393615  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.395607  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.398525 10861 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.400377  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.401558  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.405072  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.406210  9264 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:56.407613  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.408200  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.408864  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.411610  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.412787  9264 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:56.413810  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.414783  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.415289  9267 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:56.413959  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:56.418241  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 80347.2us
I20260709 13:03:56.420614  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.421562  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.421504  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.426864  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.429813  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.430544  9858 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
I20260709 13:03:56.432041  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.436805 10854 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.437992 10854 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:56.437554  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:56.451115  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.452849  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.454582  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:56.457913  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 62257.8us
I20260709 13:03:56.466193  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:56.467657  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.468076  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:56.469163  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.470381  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.471762  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.472856  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.473078 10853 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.474053  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.473927 10884 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.474180  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.477445  9271 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:03:56.496625  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.498685  9267 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:56.474875  9266 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:56.505101 10884 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.505703 10851 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:56.506260 10851 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.508410 10851 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.508980  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.504932 10836 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.524379  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.471544  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.532189  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.534425  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:56.537956  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:56.541961  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 122054us
I20260709 13:03:56.545487  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.549139  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.545395  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:56.550601  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:03:56.551054  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 86059.4us
I20260709 13:03:56.552953  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.557909  9089 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:56.559067  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.546517 10851 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:56.562726  9828 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.562075  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.561600  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:56.563409  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.543147 10887 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:56.566574  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:56.567284  9264 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:56.568217 10884 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:56.576177 10862 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:56.560290 10857 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:56.584607  9089 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:56.561882 10867 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:56.587929 10857 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:56.561753  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:56.588135 10867 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:56.569743 10836 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.590250  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:03:56.631814  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 47330.9us
I20260709 13:03:56.631825  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.633266  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:56.566728 10860 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.672044  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.802954  9191 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:56.673779  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.637284  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:56.644915  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.673311  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.656354 10832 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:56.812510  9115 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:56.672240  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.663214 10862 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:03:56.667685  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 105040us
I20260709 13:03:56.703037 10853 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:56.839432 10832 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:56.839694  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.840067 10853 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:56.840480  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.841023  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.830283  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.841998  9194 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:56.829947  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.830945  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.843163  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.843619  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:56.844271  9191 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:56.845309  9263 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:56.846941 10860 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.588706 10857 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:56.678227  9087 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.654s (SENT)
I20260709 13:03:56.671798  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.786823 10893 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:03:56.817232  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 180257us
I20260709 13:03:56.853492 10893 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:56.855343  9825 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.855980 10893 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:56.857609 10862 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:56.858682  9162 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.857707  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:56.581998 10884 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:56.859905 10884 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:56.861334  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:03:56.862696 10863 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:56.863065 10863 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:56.863145  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:56.863654  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:56.863452 10863 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:03:56.654989  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:56.770483 10891 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:56.572379 10851 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.871594 10891 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.868700  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:56.877451  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:56.880744 10891 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:56.881532 10891 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:03:56.884765  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 293891us
I20260709 13:03:56.885480  9162 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.888410  9271 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:56.894371  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.887202  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:56.896111  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:56.643394 10869 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:56.586390 10838 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:56.769935 10894 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:56.825423 10867 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:56.906023  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.899719  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.884212 10891 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:56.743530  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:56.904099  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:56.887420 10852 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:56.887669 10895 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:56.897819 10874 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:56.922237  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.586397 10887 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.900493 10860 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:03:56.928840  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 51669us
I20260709 13:03:56.900802 10869 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:56.890028 10854 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 13:03:56.946720  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 47093.7us
W20260709 13:03:56.950979  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 95751.3us
I20260709 13:03:56.917562 10891 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:56.952545  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:56.952718  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 79992.1us
I20260709 13:03:56.953913  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:56.923628 10893 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:56.902190 10864 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:56.949525  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.926985 10851 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:56.946825 10858 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:56.924291 10895 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:56.956677  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:56.968621  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.975106 10895 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:56.977578  9828 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:03:56.947187  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 269028us
I20260709 13:03:56.947121 10860 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:56.951195 10869 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:56.955909 10866 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:56.907111 10894 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:56.964593 10864 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:56.965147 10893 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:56.986652  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:03:56.992659  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 86703.7us
I20260709 13:03:56.980293 10901 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:56.978770 10874 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:56.980023 10860 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:56.979813 10853 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:57.036536  9162 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:56.992177 10864 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.000145 10893 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:56.990896 10902 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.041208  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.012943 10901 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:57.042484  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 89051.5us
I20260709 13:03:57.008646 10874 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:03:57.094589  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 120180us
I20260709 13:03:57.107206  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:03:57.108896  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 203003us
I20260709 13:03:56.987910 10858 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 13:03:57.117787  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 76695.5us
I20260709 13:03:57.118855  9162 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.103871 10874 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.120134  9162 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:57.120977  9826 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.106550 10892 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.135299  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.143033  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.146628  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.143482  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.143682  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:57.166910 10916 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.167068  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:57.169235  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.190462  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.191864  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.150079 10898 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:57.194669  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 73810.2us
I20260709 13:03:57.154371 10832 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:57.155217 10888 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:57.200047  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:03:57.205991  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 98926.8us
I20260709 13:03:57.168123 10853 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:57.207950  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.162807  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:03:57.212081  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49419.2us
I20260709 13:03:57.217120  9087 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:03:57.222590  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 79193.9us
I20260709 13:03:57.193722 10866 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:57.223390  9164 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.232095  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.215415 10853 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:57.198827 10832 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:57.208520 10888 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:57.209918 10892 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.234476  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.261715  9087 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:03:57.234472  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 122769us
I20260709 13:03:57.210867 10916 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.201704 10900 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.142287 10910 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.249635 10860 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.250051 10864 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.276921  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.251152  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.219990 10865 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:57.252620  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:57.218662 10871 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.225235 10905 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.210822 10906 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:57.224244 10887 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:57.257397 10884 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.228463 10866 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:57.234706 10897 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.291889  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:57.272874 10837 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.269835  9087 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.242069 10836 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.224152 10891 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.252369 10872 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.298835  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.197273 10911 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:03:57.259372 10832 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:57.255975 10853 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.261272 10888 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.284888  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.289429 10894 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:57.216145 10902 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:03:57.274142 10889 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.289860  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:57.300179 10837 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.300302  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.303673  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.288408 10906 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:57.288602 10892 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.317749  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.342825  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:03:57.345556  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 153756us
I20260709 13:03:57.345980  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.287566 10880 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.293677 10866 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:57.333217  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.334344  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.335107  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.352571  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.335831  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.336078  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.336705 10910 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:57.336844  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.337134  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.346246  9162 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:03:57.349162  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72164.1us
I20260709 13:03:57.353158  9859 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
W20260709 13:03:57.353603  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 71299.7us
W20260709 13:03:57.357340  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 79776.1us
I20260709 13:03:57.382769  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:57.340610  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.343046 10906 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.358407  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.382413  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:57.387964  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.385699  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.394071  9162 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.359365  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:57.395879  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.396310  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.400744  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:03:57.400804  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 131098us
I20260709 13:03:57.401751  9087 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.406635 10924 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.382880 10860 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.410590  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.358659 10892 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.415877  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.402926 10852 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:03:57.405454 10910 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.410934 10880 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.382434 10926 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.427263 10917 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:57.430990 10891 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:57.432693 10891 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.433636 10877 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.434419 10903 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.425899 10892 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:57.444157 10892 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:57.444579 10892 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:57.449741 10854 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:57.422346 10852 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:57.451776 10854 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:57.455461 10919 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:57.461566 10893 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.462031 10921 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:57.456895 10869 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:57.471956 10921 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:03:57.456895 10857 consensus_meta.cc:287] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.609s	user 0.006s	sys 0.005s
W20260709 13:03:57.475484  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 81500.8us
I20260709 13:03:57.476274  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.459091 10852 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:57.464428 10863 consensus_meta.cc:287] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.600s	user 0.008s	sys 0.004s
I20260709 13:03:57.453737 10864 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:03:57.477316  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 76639.4us
I20260709 13:03:57.478729  9162 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.487474  9162 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.487515  9123 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.491667  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.474023 10884 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:03:57.498327  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 95221.6us
I20260709 13:03:57.499255 10904 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:57.499404  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:57.477458 10857 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.508962 10857 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.509624 10923 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.510550  9119 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:57.505198 10869 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:57.478807 10926 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:57.515187  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 274796us
I20260709 13:03:57.484462 10909 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.457505 10917 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.520984  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.521143  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:57.520426  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.523180  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.525206  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.526896  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.503386 10888 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.532413 10925 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.488171 10863 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.530777  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.523897 10864 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.532356  9116 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.536243 10888 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.536558 10837 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.537319 10931 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:57.538009  9114 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.528332 10860 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.522167  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.549520 10893 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.551990 10865 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:03:57.574970 10865 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.562469 10917 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:57.577677 10874 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.523447  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.552747 10921 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:57.558501  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.557477 10866 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.588560  9189 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:57.559121 10880 raft_consensus.cc:1064] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:57.523391 10909 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:57.551918  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.494349 10919 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.561527  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.562289  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.561205  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.575438 10865 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:57.571854  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:57.572423  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 156658us
I20260709 13:03:57.572458  9263 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:57.572603  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:03:57.595831  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 194230us
I20260709 13:03:57.582551  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.585043 10863 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.596863 10921 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.598182  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.587359 10892 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.568825  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.600844 10921 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:57.592415 10932 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.602945 10921 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.603319 10921 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:57.594094 10880 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.594647 10906 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:03:57.596282  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 44449.5us
I20260709 13:03:57.540743 10858 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.612205 10865 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:03:57.575155 10867 consensus_meta.cc:287] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.655s	user 0.008s	sys 0.003s
I20260709 13:03:57.599231  9087 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.600289 10911 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.598403 10907 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.596561 10931 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.620114  9190 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.620170 10857 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.620390 10877 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.601598  9120 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:03:57.581621 10895 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.001s
I20260709 13:03:57.581910 10874 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.607913  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.609131 10919 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.614154  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.613430 10865 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.615440 10909 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:57.634145 10931 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:57.634280 10909 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.616178 10854 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.618142  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.625582  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.625372 10895 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:57.627089 10892 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.629427  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.630934 10886 raft_consensus.cc:993] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:57.631242  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:57.618050 10908 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.638772 10906 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.619474 10866 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.638274  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.640483 10921 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.649097  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.630930  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.653318 10886 raft_consensus.cc:1081] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:03:57.642448  9116 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.652621  9196 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:57.653018 10908 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:57.651770 10854 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:57.654711 10921 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.644961 10860 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:57.658941  9238 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.654932 10908 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.660002 10908 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.621219 10867 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:57.661192  9826 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.666445  9087 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.667055 10931 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.668653 10886 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:57.636794 10858 consensus_queue.cc:1175] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ce51b61da22f40d4aa0204a6b1028f28" responder_term: 2 status { last_received { term: 2 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 2 index: 3 } }
I20260709 13:03:57.662133 10860 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:57.671631  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.673822 10860 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:57.660029 10863 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:57.674325  9118 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.663898 10916 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.679164 10916 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.676640 10863 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:57.666304 10864 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.682353  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:57.688539  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.689500  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:57.685583 10863 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:57.651217 10919 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:03:57.690091  9193 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:57.698980  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.700954 10895 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:57.690697 10905 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.701757 10905 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:57.694320  9192 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.674500 10931 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:03:57.688990  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.704663  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.705456  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:57.706713  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.702582 10905 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.707641  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.708240  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:57.708801  9195 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:57.709045  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.711432  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.712062  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.712817  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.716792  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.717307  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.722728  9267 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39754
I20260709 13:03:57.723265  9267 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:03:57.723611  9267 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:57.724771  9270 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:57.725903  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.725991 10895 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:57.726966 10895 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.727300 10895 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:57.727334  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.696175 10867 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:57.728978  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.732954  9270 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.730583  9268 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.731799 10895 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.732604  9269 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.735078  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.741401  9270 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:57.737249  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.742169  9268 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
W20260709 13:03:57.735791  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46854us
I20260709 13:03:57.744606  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.745141  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.749930  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:57.750684  9119 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:57.751873  9861 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:57.755172  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.759145  9162 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.762624  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.763289 10897 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.763772  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:57.728940 10832 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:57.764590  9115 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:57.760455  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.767278  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.769785  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.770845 10866 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:57.771255 10866 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:57.771654 10866 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:03:57.773852  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.695780 10852 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.737206 10901 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.780418  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.782661 10874 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.785000  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.785840 10858 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.786253 10858 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:57.787068 10858 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:57.780098  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:57.789103 10857 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.790165  9123 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.789218 10895 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.795276  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:57.795899  9123 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:57.796378 10900 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.796795 10900 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:57.799803  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.797580 10900 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.802963  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:57.800817  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.804383  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.806092  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.808981 10890 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.811014 10905 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.814255  9162 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:57.815006  9267 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:57.782330 10860 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:57.829699  9272 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:57.827389 10893 raft_consensus.cc:1064] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:03:57.828154 10906 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:57.817019  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:57.835403  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.817273 10887 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:03:57.846530  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.847635 10901 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.822355 10852 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.822046 10912 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.838812  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.803254 10874 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.839998  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.763831 10919 consensus_queue.cc:1059] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:03:57.860267  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.845695  9267 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:57.816831 10907 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.847692 10909 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:03:57.862138  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.849191  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:57.849520  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:57.865036  9123 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:57.836637  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.865370  9123 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:57.732980 10853 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.855518  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:03:57.867851  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.868096  9268 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:57.857137  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.857940 10890 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:57.848224 10932 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
W20260709 13:03:57.843932  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.855330 10858 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.855690 10931 consensus_queue.cc:1059] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:03:57.868249  9162 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.867569 10916 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:57.866159 10917 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:57.856130 10904 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:57.884145 10904 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.874065 10887 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:57.884542 10904 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:57.862140 10832 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:57.862257 10909 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:57.879091  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:57.880138  9197 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.883661 10917 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.888972  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:57.872125 10903 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.892782 10893 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.887229  9267 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:57.845625  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.872118 10900 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.897970 10914 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.879216 10933 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:03:57.881754 10890 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:03:57.900439 10917 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:57.906157  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:03:57.906793  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 61258.6us
I20260709 13:03:57.904628 10896 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:03:57.884630 10887 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:57.912289  9191 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.901124 10909 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.914045 10892 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:57.918984  9195 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:03:57.910727  9123 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:57.914986  9238 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.924535  9123 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:57.926515  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.927814  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:57.909835 10864 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.912470 10910 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:57.918474 10920 raft_consensus.cc:3055] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.929976 10925 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.922559 10887 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.910346  9118 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:57.934631 10854 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:57.937409 10920 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:57.937321 10902 consensus_queue.cc:1059] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:03:57.940284  9114 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:57.918665 10914 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:57.957386  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:57.928998 10865 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.965909 10896 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:57.967383 10894 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.939579 10907 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:03:57.954602 10854 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:57.977486 10854 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.978721 10854 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:57.975551 10933 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:57.980067 10933 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:57.981031 10933 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.982270 10933 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:57.985591 10912 raft_consensus.cc:3055] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:57.986073 10912 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:57.986605  9114 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:57.988597 10917 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:57.989758 10916 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:03:57.997329  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:57.997874  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:58.005851  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 79052.3us
I20260709 13:03:57.969501 10896 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:03:58.012104  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:57.960057 10860 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:03:58.014000  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:58.005260 10916 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:58.022610 10916 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:58.022993  9114 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:58.023527 10916 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:58.024237 10902 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:58.025049 10916 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:57.969030 10886 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.025447 10897 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:03:58.029343  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 101654us
I20260709 13:03:58.031339  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.971906 10894 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:58.034832  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.035166  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:58.035965  9268 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:03:58.036041  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.964864 10864 consensus_queue.cc:1059] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:03:58.037243  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.017594 10853 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:58.042933  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:57.990239 10920 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:58.060129  9191 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
I20260709 13:03:58.026727 10911 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:03:58.038673 10860 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:58.039139 10865 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:58.039323 10907 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:57.939015 10910 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.044528 10874 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:58.043684 10902 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 13:03:58.079113 10919 consensus_queue.cc:1175] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "92919b70789041c8a0de0b3525154fd1" responder_term: 2 status { last_received { term: 2 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 2 index: 3 } }
I20260709 13:03:58.105484 10866 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:58.081676 10862 raft_consensus.cc:3055] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:58.082953 10864 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.123297  9191 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
W20260709 13:03:58.126734  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45697.9us
I20260709 13:03:58.142668  9195 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:58.090175 10896 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.144881 10896 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:58.091514 10888 consensus_queue.cc:1059] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:03:58.116384 10862 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:58.148692  9268 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:58.089810 10894 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.166018 10894 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.160539 10903 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:58.140772 10921 consensus_queue.cc:1059] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:03:58.165081  9123 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:58.161221  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.187462  9235 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:58.188562  9087 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:58.200111 10860 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.216957  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.218940  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:58.220811 10922 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:58.221365 10901 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.221657 10922 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.222466 10910 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:58.223691  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:58.227283  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.223943 10864 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:58.228428 10864 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:58.228909  9826 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.146651 10909 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.229370 10864 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:58.230960 10909 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:58.231041  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.232414 10901 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.233132 10901 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.233791 10901 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:58.234189 10901 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:58.234572 10901 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.236304  9267 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:58.237300 10931 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:03:58.224164  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.237711  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:58.224418 10889 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.240847 10931 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:58.242895  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.225257  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:58.243256  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.244050  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.244138  9087 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:03:58.244467  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.245087  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.249054 10931 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.249557  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.250308 10867 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.250607 10931 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:03:58.224097 10863 consensus_meta.cc:287] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.001s
I20260709 13:03:58.152786 10932 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:03:58.247663  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.257189  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:58.225057 10866 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:58.267441 10789 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:03:58.267882 10789 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:58.268718 10789 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.269596 10862 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.270926 10862 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:58.251680 10902 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:58.274909  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:58.276782  9858 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:58.251567 10889 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.260701 10863 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:58.277247  9197 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.252053 10930 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.252321 10908 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:03:58.283659  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.278290  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:03:58.290678  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.282706  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:58.258325 10921 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.284595 10914 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.291711 10937 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.294674  9853 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:58.297848 10866 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:58.301054  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:58.303341  9114 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:03:58.304150  9197 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:58.303335  9120 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:58.304364  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.274142 10940 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:58.279126 10902 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:58.310365  9264 raft_consensus.cc:1240] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:58.272143 10888 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:03:58.300226 10911 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.170287 10912 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.317369  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.315271 10902 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:58.305449 10932 raft_consensus.cc:3055] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:58.276105 10904 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.306380 10933 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:03:58.325572  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 81181.5us
I20260709 13:03:58.323154 10872 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.326705  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.338883  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:58.339790  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.350322  9859 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:03:58.360160  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.314693 10908 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.330328 10789 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:03:58.373270  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 51261.7us
I20260709 13:03:58.322577 10932 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:58.376154  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:58.332234 10917 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.364849 10858 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.381248 10932 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.362318 10914 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.364529 10874 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.391621  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.399972  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:58.378648 10866 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:58.404098  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:58.404731 10888 consensus_queue.cc:1059] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:03:58.406355 10867 raft_consensus.cc:3055] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:58.406706 10867 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:58.407606 10867 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:58.398101 10872 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:03:58.408931 10866 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:58.409901 10866 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.410892 10898 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:58.370812 10884 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:03:58.424921 10912 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.435309  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.400290  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:03:58.441447  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 59570.8us
I20260709 13:03:58.445798  9263 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:58.400261  9116 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "470326df6cae476c90888533e504e6ec"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:37642
I20260709 13:03:58.446923  9116 raft_consensus.cc:606] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:03:58.448704 10887 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.437633 10874 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.449751  9861 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:58.449177  9019 catalog_manager.cc:5697] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.442653 10890 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:03:58.450953 10905 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.639s	user 0.005s	sys 0.005s
I20260709 13:03:58.450657 10890 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:03:58.452395  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.453868  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:03:58.454356  9862 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:58.455529  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:03:58.434122 10837 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:03:58.459444  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:58.464010  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.464087  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:58.464866  9119 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:58.465668  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:58.468678 10863 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:58.470185 10904 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:58.472062  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:58.472163  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:58.472632  9119 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:58.441215 10884 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.462153  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:58.474671  9115 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:58.480373  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:58.480981  9861 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:58.463119 10837 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:03:58.487696  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45994.1us
I20260709 13:03:58.462800 10901 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.490031 10837 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:58.491062 10901 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:58.491457 10837 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.496398 10887 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:58.458026 10890 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:03:58.497785  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.501737  9861 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:03:58.503120  9119 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:03:58.504108 10920 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.506651 10902 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.509084  9164 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:58.517647  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:58.509512  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:58.519819  9119 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:58.559813  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:58.568439 10908 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.515615 10838 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:58.561635 10904 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:58.591630  9859 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:58.596340 10904 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:58.594693 10838 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:58.634727 10838 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:58.637926  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:58.640939  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:58.638573  9272 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:03:58.636649 10832 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:58.644079 10832 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:58.644304  9192 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:58.644964 10832 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:58.648988  9087 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:58.649832 10880 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:58.650213 10880 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:58.650563 10880 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.653438  9271 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:58.654487  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.655654 10917 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:58.656211 10917 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:58.657073 10917 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.660748  9266 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:03:58.660552 10904 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.664640 10932 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.655790 10903 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:03:58.671511 10903 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:03:58.675843  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.678795  9263 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:58.679482 10902 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:58.680830  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:58.683070  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:58.681329 10789 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:03:58.672395 10903 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.686569  9825 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.686964 10903 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.691898 10789 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:58.692523 10789 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:58.693688 10789 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.698761 10789 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:58.688390 10902 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:03:58.712450 10904 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:58.716526  9853 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:58.729895 10880 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:58.736725  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1446) took 1020 ms. Trace:
I20260709 13:03:58.737190  9266 rpcz_store.cc:276] 0709 13:03:57.715957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:57.724005 (+  8048us) service_pool.cc:224] Handling call
0709 13:03:58.736564 (+1012559us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54002a4c54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:03:58.736668 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199823744}
I20260709 13:03:58.741214  9162 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:58.742890 10853 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:58.743296 10853 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:58.743688 10853 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.746843  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.748723 10867 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.753383  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:58.754245  9190 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:58.738708  9020 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 3, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.755493 10880 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:58.756174 10889 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:58.759507 10889 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:58.762815  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.764570  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:58.761873 10889 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:58.768100 10895 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:58.768558 10895 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:58.768918 10895 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.790575  9190 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:03:58.791015  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:58.776947 10853 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:58.798815  9264 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:03:58.802712 10858 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:03:58.812726 10853 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:58.816808 10832 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.820562  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:58.824092  9264 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:58.824285  9087 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.825665  9114 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
W20260709 13:03:58.828280 10952 log.cc:927] Time spent T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Append to log took a long time: real 0.053s	user 0.000s	sys 0.004s
I20260709 13:03:58.831714  9268 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:58.832732  9162 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:58.833787 10872 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:58.834244 10872 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:58.848032  9235 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.838513  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:03:58.845342  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:58.851944  9114 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:58.835047  9268 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:58.863471 10858 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:58.868036 10858 raft_consensus.cc:697] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:58.852514 10838 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:58.838482  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:58.852496 10909 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.835098 10872 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:58.871361 10858 consensus_queue.cc:237] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:58.863718 10893 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:58.882249  9017 catalog_manager.cc:5697] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.881201  9118 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:58.888162 10893 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.889617 10865 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:03:58.889429  9116 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:58.890673 10893 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:58.889469  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.892970 10896 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:03:58.893784  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:58.894440  9855 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:58.838152 10895 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.895725  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:58.896790 10854 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:58.897131 10854 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:58.897503 10854 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:58.897552 10896 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:58.897825 10895 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:58.899145  9268 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:58.877668 10838 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:58.901840 10838 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:58.904690 10893 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:58.905478  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:58.898104 10896 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.913728  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:58.914469  9119 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:58.926661  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:58.927699  9114 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:58.939615  9853 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:58.941682 10896 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:58.942411  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:58.943377  9119 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:58.949321  9855 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:58.950417  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:58.951120 10872 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:58.951864  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:58.952512  9190 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:58.960414  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:58.962177 10902 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:58.962633 10902 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:58.963042 10902 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.964978 10872 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:58.968218 10854 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.971028 10872 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:58.987809 10896 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.988165 10854 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:58.981436 10895 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:03:58.992153 10902 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.005239  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:59.004240 10896 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.007702  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:59.023149  9018 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:59.025563  9018 catalog_manager.cc:5697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 2 to 3, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:59.024266  9116 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:59.069106  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:59.069358 10902 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:59.069907  9119 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:59.071188  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:59.070320 10865 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.077788 10865 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.080076 10865 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:59.081813  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.087116  9119 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:59.088070  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:59.088162  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.089473  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.089918 10858 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:59.091272  9190 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:59.091373 10858 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.091576  9197 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:59.091406  9116 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:59.094619 10858 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.095726  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:59.096938 10917 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:03:59.097434 10917 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:59.098306 10917 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:03:59.100347 10959 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:03:59.101189  9825 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:59.101791 10896 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.103569  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.104259  9264 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:59.105463 10858 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.108789  9853 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:59.110231  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:59.111168 10893 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:59.111670 10893 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:59.112557 10893 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.116117  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:59.116731  9116 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.118606 10898 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.119120 10898 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:59.121021  9118 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:59.121201 10898 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.122092  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:59.122692  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.122893  9197 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.123265  9268 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:59.125586  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.126698  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:59.128968  9197 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:03:59.129496 10872 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:59.129875 10872 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.130252 10872 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.134640 10902 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:59.135092 10902 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.135460 10902 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.137362  9238 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:59.141649 10896 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:03:59.107692 10858 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:59.149972 10865 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:59.130218  9116 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:03:59.174449  9268 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:59.165679 10865 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:59.177508 10889 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:03:59.185130  9853 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:59.190446 10890 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:59.190112 10865 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.194729 10898 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:03:59.195291 10898 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.198123 10898 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.179069 10872 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:59.199273  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.200431 10872 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.198356  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.200997  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:59.202538  9161 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:59.203852 10872 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:03:59.207646  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:59.208410  9264 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.217304  9271 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:03:59.220356 10896 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:03:59.223457 10932 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:59.224068 10932 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:59.224973 10919 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:59.225485 10919 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.226275 10932 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:59.227376 10919 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:59.222990  9019 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.232730  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.233672  9197 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:59.235597  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.236317  9197 raft_consensus.cc:2393] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:59.232715 10902 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.238756  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.240595  9114 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:59.241888  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:59.253750 10936 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:59.253849  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:59.254230 10936 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:03:59.254447  9114 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.254642 10936 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:59.260720  9116 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:03:59.262159 10872 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:59.263152 10936 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.232161  9019 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.266710  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:59.268635  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:59.269249  9272 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.269191 10902 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.279222  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.280519  9118 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:03:59.281728  9235 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:59.268182  9019 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 3, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.283440  9272 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:03:59.283485 10946 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:59.291379  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:03:59.294354  9197 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:59.294512  9264 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:03:59.296011  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:59.301951  9114 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:03:59.302525  9120 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:03:59.302963 10936 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:59.305711  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:59.306313  9197 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:59.307440  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:59.308226  9192 raft_consensus.cc:2393] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:03:59.307816 10890 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.309378 10889 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:03:59.309783 10890 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.313524 10919 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:59.305370 10902 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:59.324090 10902 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:59.311288 10898 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:59.325258 10902 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.325994 10898 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:59.326721  9020 catalog_manager.cc:5697] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:59.331110  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.331702  9114 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:59.326930 10898 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:59.323616 10890 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.335237  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.335904  9853 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:59.336443 10917 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.336920 10917 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.339510  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.339996  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:59.340195  9190 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:59.340771 10946 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:03:59.341126 10893 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:59.341526 10893 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.341863 10893 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.345328  9828 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:59.346179 10914 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:59.346550 10914 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.346855 10914 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.352579  9264 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:59.362294 10838 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.323628 10919 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:03:59.369596 10919 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.370926 10917 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.373451  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.373646 10934 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.374228  9264 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:03:59.374351 10934 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.365396  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "991b28944986479a8ad096ca92f1a435"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:59.377966 10837 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.389370 10837 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.363205 10889 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:03:59.394408 10889 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:03:59.394925 10889 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:03:59.396129 10889 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:59.398026 10889 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.402582 10909 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:59.403133 10909 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.405206 10909 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:03:59.406287 10934 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.406685  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.407406  9114 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 2: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:59.368091 10838 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.407855  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.408974  9856 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:03:59.409355 10838 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.412062  9190 raft_consensus.cc:606] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:03:59.417192  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:59.418982  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.418577  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.421804  9191 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:03:59.424504  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:59.430644  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.433593  9862 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:03:59.433928  9191 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "991b28944986479a8ad096ca92f1a435"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:59.434427  9191 raft_consensus.cc:606] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:03:59.394941 10837 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.421804 10888 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:59.447463 10888 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.447893 10888 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.428133  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.446362  9194 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.429256 10832 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:59.426811 10895 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.453157 10832 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.453313 10895 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.456807  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.457405  9268 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:03:59.453631 10832 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:59.458498  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:59.459091  9190 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:59.432327  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.460088 10903 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:03:59.460529 10903 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.460879 10903 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:59.422466 10893 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.462903 10895 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.462936 10914 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.464689 10893 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.465317  9268 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:59.469158  9856 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:59.466815 10854 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:03:59.470604 10832 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.472208  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:59.473635  9194 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.475797  9854 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Advancing to term 3
I20260709 13:03:59.475941  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.476616  9856 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:03:59.476987  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:59.477546  9853 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.480423  9862 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:03:59.482785 10893 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:03:59.483173  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.484790 10893 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:59.471660 10888 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.494504 10914 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.474887 10902 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.495829  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:59.496446  9268 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.496112 10902 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.496547 10888 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.497964  9854 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:59.508518 10902 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.508754 10903 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.540374  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:03:59.557462  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:03:59.558831  9017 catalog_manager.cc:5697] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.568552  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.569985  9114 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:03:59.575187  9118 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:59.577095  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:03:59.588546  9114 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.585975  9018 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:59.608851  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:59.587446  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:03:59.628083 10895 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.629105 10917 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.630090 10917 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.633921 10946 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.634944 10946 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.628188 10854 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.637040 10832 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.637595 10854 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.638799 10946 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.640049 10934 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:59.640734 10934 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.641222 10934 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.644586  9197 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:59.648032 10919 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.648876 10919 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.648876  9268 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:03:59.629101 10895 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.662745 10895 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.662878 10865 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:03:59.640165 10981 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.668390 10919 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.669100 10896 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.670101 10896 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.672490 10946 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:03:59.672809  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 85921.8us
I20260709 13:03:59.673061 10946 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:03:59.673652 10946 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:59.675128 10903 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:03:59.677723  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 120737us
I20260709 13:03:59.678229  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.679209  9854 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:59.680850  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:03:59.664382 10854 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.668432 10981 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.693590 10931 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:59.694298 10981 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.694592 10931 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:03:59.696673  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.697707  9272 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:59.698683  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.698957 10917 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.699625  9263 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:59.678901  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.701040  9862 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:59.703958  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:03:59.723492  9263 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:59.728755  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.729687  9115 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:03:59.732923 10872 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:59.740648  9194 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
W20260709 13:03:59.760411  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 78218.4us
W20260709 13:03:59.769429  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:59.780506 10931 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.783702  9115 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:59.766413 10896 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.784724 10893 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.785506 10893 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.790441  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:59.790961  9235 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:59.792377  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:59.793242 10865 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:59.793437  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:59.793759 10865 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.794217 10865 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:59.798282 10890 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:59.798511  9194 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:59.805227  9114 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:03:59.809717  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.811043  9825 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:59.800915 10890 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.812067 10890 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:59.812254 10917 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:59.812670 10917 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.813023 10917 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.801301 10919 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:03:59.803448 10893 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.820340  9197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "991b28944986479a8ad096ca92f1a435"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:03:59.821148  9197 raft_consensus.cc:606] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Received request to transfer leadership
I20260709 13:03:59.822193  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.822947  9272 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:03:59.824560  9268 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:59.810153 10928 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:03:59.825419 10928 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.825842 10928 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:03:59.830116  9118 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:59.832191  9264 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:59.836258  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.836588  9118 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:03:59.837108  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:59.837675  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.838449  9855 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:03:59.838945  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.839646  9195 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:03:59.840067  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:03:59.840930  9192 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:59.844092  9197 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:03:59.847097  9120 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:03:59.858206  9263 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:03:59.859952 10936 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:03:59.855640  9194 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:03:59.897882  9192 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
W20260709 13:03:59.908558  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 228056us
I20260709 13:03:59.882692 10889 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.897481  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:59.872944  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.837132  9189 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:03:59.934953  9114 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:03:59.877205  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:03:59.892571 10919 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:59.943216  9855 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:03:59.934720 10898 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:03:59.945115 10895 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:03:59.947479 10989 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:03:59.878193  9853 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:03:59.877951  9856 raft_consensus.cc:2955] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:03:59.939643  9858 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:03:59.935431  9114 pending_rounds.cc:85] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 3
I20260709 13:03:59.948851  9114 pending_rounds.cc:107] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:03:59.949225  9114 raft_consensus.cc:2889] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:59.876832  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:03:59.943300 10919 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.937148 10838 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.909574 10990 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.940819 10889 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:03:59.882530 10934 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.894483 10914 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.948084 10989 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.955792  9862 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:03:59.957423 10919 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.955022  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:03:59.928045  9018 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 3, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.863690 10917 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.961438  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:03:59.966583  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:03:59.908273 10946 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.967449  9114 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:03:59.898203 10890 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.964936 10989 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:03:59.965075 10896 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.971015  9018 catalog_manager.cc:5697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 3 to 4, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:59.978875  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 45206.4us
I20260709 13:03:59.981904 10854 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.982295  9116 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:03:59.982447 10854 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:03:59.974866 10838 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.972152 10858 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:03:59.972872 10990 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.973858  9854 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:03:59.983683  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:03:59.990209 10893 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:03:59.990777  9263 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:03:59.991298 10914 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.985546 10909 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:03:59.992686 10854 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.985421 10991 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:03:59.993453 10914 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:03:59.993513 10991 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:59.990816 10893 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:03:59.996402 10991 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.986433  9194 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:03:59.968959 10936 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:03:59.988488 10889 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:00.007197  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:03:59.996960 10893 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.986678 10990 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:00.008082  9194 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:03:59.985949 10838 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:03:59.997864 10928 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:03:59.990031 10896 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.013717 10890 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:00.014637  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 82093.1us
I20260709 13:04:00.015092 10896 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:00.016832  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:00.017446 10934 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:00.017606 10928 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:00.018128 10936 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.019706  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:00.020259 10917 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:00.020344 10936 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:00.020653  9859 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:00.021373 10990 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:00.024367 10988 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:04:00.018338 10928 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:00.025436 10990 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:00.040493  9069 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:04:00.041092  9069 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 991b28944986479a8ad096ca92f1a435 already in progress
W20260709 13:04:00.041477  9069 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 991b28944986479a8ad096ca92f1a435 already in progress
I20260709 13:03:59.998767 10931 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:00.064150  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:00.066712  9828 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:00.066854  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:00.067256  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:00.068828  9859 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEARNER]: Advancing to term 4
I20260709 13:04:00.071313  9854 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:00.068094  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:00.085691  9862 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:00.068367  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:00.068451 10896 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:00.067525  9858 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.056468 10889 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:00.079792  9859 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:00.072022  9197 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:00.080680 10946 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:00.081130 10931 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.074682 10865 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.038018 10990 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:00.083858  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 67179us
I20260709 13:04:00.061694 10917 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
W20260709 13:04:00.089561  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 108920us
I20260709 13:04:00.095294  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:04:00.089110  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 105262us
I20260709 13:04:00.087462 10896 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.104682  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:00.105002  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.105315  9271 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.105616  9263 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:00.101200 10872 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:04:00.106738 10872 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:00.107108 10872 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:00.108013  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 98d913b72e5b43e8b8f20fb99d0838c4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:04:00.108139 10872 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.109426 10872 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:00.112244 10990 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.113546 10990 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:00.114094  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:00.114717  9266 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:00.099390 10988 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:04:00.117368 10932 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:00.117820 10932 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.118188 10932 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.120095 10865 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:00.120513  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.121109  9264 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:00.122008 10917 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:00.122398 10917 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:00.079402 10989 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.128504  9271 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:00.127290  9862 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:00.130415  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:00.136238 10931 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:00.136915 10991 raft_consensus.cc:993] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:04:00.101759 10896 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:00.138029  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:00.139490  9858 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:00.141654 10838 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:00.143189  9862 raft_consensus.cc:2368] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4.
I20260709 13:04:00.146986  9194 raft_consensus.cc:2393] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:00.149503  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:00.144367 10991 raft_consensus.cc:1081] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:04:00.150482  9116 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 2: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:00.164546  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.162433  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:00.169377  9859 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.171319  9192 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:00.179855 10989 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:00.181926  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.182636  9264 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:00.183534  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.184549  9114 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "991b28944986479a8ad096ca92f1a435"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43782
I20260709 13:04:00.179935  9265 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:00.186558  9271 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:00.166209  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:00.193095  9120 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.187589  9087 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:00.160985  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:00.203214  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:00.156776 10838 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:00.128507 10917 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.156286  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:00.164625  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.217612  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:00.218200  9271 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:04:00.224371  9859 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:00.170437 10893 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:04:00.215981 10889 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:00.215493  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:00.308925  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 65492.8us
I20260709 13:04:00.309114  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:00.309788  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:00.310325  9191 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:00.312189 10838 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.315588  9196 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:00.230336  9263 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:00.321283  9164 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:00.267141 10917 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.266708 10889 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.349000  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:00.371987 10889 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:00.372318  9855 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:00.350033 10890 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:00.373988 10890 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:00.351439 10988 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:00.376158 10988 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:00.377316  9116 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:00.377583 10988 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.379772 10890 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.351907 10920 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:00.358398  9017 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.360085  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:00.362288 10896 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.364114  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.365199  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:00.275585 10832 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.370684  9121 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:00.417204  9017 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.424491  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:00.389326  9162 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:00.415552  9196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:04:00.395680  9271 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:00.426707  9196 raft_consensus.cc:606] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Received request to transfer leadership
I20260709 13:04:00.441169  9118 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:00.428965  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:00.421638 10893 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:00.466245  9265 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:04:00.468178  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 146985us
I20260709 13:04:00.486567  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:00.487226  9857 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:00.490196  9120 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:00.505420 10932 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.506526 10872 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:00.478734 10928 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.516026  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.499521  9856 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:00.485859  9263 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:04:00.507193 10872 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:00.450258 10991 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:04:00.531243 10932 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:00.479513 10832 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:00.448288 10920 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.532481 10872 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.508013 10893 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:00.555387  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 193239us
I20260709 13:04:00.540263 10865 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.555419 10872 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:00.536119 10928 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:00.612525  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.613940  9268 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:00.634908  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.591594 10865 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:00.647713  9266 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.532s	user 0.011s	sys 0.003s
I20260709 13:04:00.657320 10896 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:00.671298 11021 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:00.661259 10858 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.024s	sys 0.075s
W20260709 13:04:00.675086  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 285869us
I20260709 13:04:00.682714 10893 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:00.682987  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:00.684378  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:00.668623  9266 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:00.677683 10858 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.024s	sys 0.075s
I20260709 13:04:00.712427  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.713646  9272 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 2: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:00.723083 10992 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:00.733111  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 309354us
W20260709 13:04:00.749297  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 181341us
I20260709 13:04:00.739110 11027 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.759563  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47352.1us
I20260709 13:04:00.761101  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:00.765053  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:00.741619 10920 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:00.771183  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:00.797394  9238 leader_election.cc:400] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4.
I20260709 13:04:00.797813  9238 leader_election.cc:403] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:00.799412  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 115168us
I20260709 13:04:00.799991  9120 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:00.800437  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:00.762063  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:00.803102  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 54684.6us
W20260709 13:04:00.772639  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.805087  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:00.807382  9122 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:00.807284 10928 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:00.770172 10832 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:00.807227 10896 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:00.806910  9195 raft_consensus.cc:2410] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:04:00.819473  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 55302.7us
I20260709 13:04:00.819921  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:00.815241  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:00.827292  9118 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:00.805697 10920 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4.
I20260709 13:04:00.739718 11021 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.811568  9854 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.816232 10896 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.817574 10865 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
W20260709 13:04:00.868942  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 46778.6us
I20260709 13:04:00.776520 11027 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:00.872897  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 101174us
I20260709 13:04:00.883109  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:00.876852 10896 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:00.891044 11017 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:00.883875 10997 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:00.804662 10992 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
W20260709 13:04:00.912561  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:00.911525  9119 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:00.913795  9857 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:00.913933  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:00.915493  9114 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:00.918340  9114 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:00.918256  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:00.872164 10917 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:00.920382  9196 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:00.920867 10917 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:00.921923 10917 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.923911  9855 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:00.924872  9265 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:00.925859 10914 raft_consensus.cc:2764] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:04:00.917127 11017 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.928133 11017 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:00.929126  9271 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:00.929911  9857 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:00.930511  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:00.933770  9194 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:00.934296  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:00.934947  9272 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:04:00.942874  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:00.943671  9189 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:00.944172 11025 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:00.946630  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:00.940016  9164 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:00.948426  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:00.949158  9116 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:00.953538  9114 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:00.997005 11036 raft_consensus.cc:993] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:04:00.997802 11036 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:00.999931 10992 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:01.003649 10992 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.005719 11019 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:01.007068 10999 raft_consensus.cc:1081] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:04:01.006927 10896 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:01.009629 11011 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.340s	user 0.001s	sys 0.000s
I20260709 13:04:01.015851 10992 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:01.017515  9189 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:01.021401  9854 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
W20260709 13:04:01.020927  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 67638.9us
I20260709 13:04:01.025260 11027 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:00.959559 10997 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:01.027287 10997 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:00.943470 11021 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:01.033771  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:01.038174  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 102514us
I20260709 13:04:01.039696 11040 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 13:04:01.002455 10889 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.003s
I20260709 13:04:01.046984  9264 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:04:01.048336 11034 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:04:00.937181 10872 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.050571  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:01.060513  9272 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:01.082676 10858 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:01.085557 10896 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:01.088076  9192 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:01.087652  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:01.088687  9116 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:01.089279  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:01.067626 10889 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.061129 11017 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.094278 11017 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:01.097609  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 146713us
I20260709 13:04:01.098742  9119 raft_consensus.cc:2368] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5.
I20260709 13:04:01.101372  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:01.101752  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 51291.9us
I20260709 13:04:01.102201  9858 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:01.102869  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:01.107211  9857 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:04:01.138219  9264 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:01.139542  9857 pending_rounds.cc:85] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 4
I20260709 13:04:01.139622  9264 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:01.139914  9857 pending_rounds.cc:107] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:04:01.140218  9857 raft_consensus.cc:2889] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:01.135854  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:01.163126  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:01.195241  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 59529.4us
I20260709 13:04:01.217163  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:01.217697  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:01.219045 10992 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:01.221451 10992 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:01.224342  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:01.225168  9857 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Advancing to term 6
I20260709 13:04:01.244596  9089 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:01.153633 10858 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:04:01.249955  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 298933us
I20260709 13:04:01.226178 11036 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:01.242110  9263 consensus_meta.cc:287] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.698s	user 0.015s	sys 0.003s
I20260709 13:04:01.251284  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:01.150467 11017 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:04:01.163992 10999 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:01.164314 10889 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:01.223485 10936 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:01.256122  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:01.222162 10992 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:01.256372  9268 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:01.257189  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:01.251523 11021 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:01.254245  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:01.258337  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:01.258044  9263 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:01.260190 11044 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:01.256446 10858 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:01.279119  9264 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:01.290589 10872 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:01.293159 11033 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:04:01.294075  9116 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:01.270694 10936 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:01.296345 11032 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:01.297986 10936 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:01.298557 11039 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.195s
I20260709 13:04:01.300575 10992 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.304618  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:01.305796 10931 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:01.306233 10931 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:01.306602 10931 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:01.307020 10858 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:01.307844 10991 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:01.304632  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:01.313031  9268 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:04:01.317849  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:01.318670  9858 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
W20260709 13:04:01.319679  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 63646.5us
I20260709 13:04:01.312772  9825 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:01.320331  9089 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:01.323437  9857 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEARNER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:01.324680  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:01.329353 10909 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.329264  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:01.330142  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:01.330049 10909 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.330862  9193 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:01.330154  9190 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:01.333874 10909 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:01.334540 11042 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:01.321063  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:01.335381  9269 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:01.335130 11042 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.326563 10896 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:01.336819  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 559) took 1290 ms. Trace:
I20260709 13:04:01.337370 10896 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:01.338384 10896 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.327778 10936 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:01.341102 10936 raft_consensus.cc:697] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:01.342190 10936 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.343937 11036 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.348065 10837 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:01.348795 10837 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:01.349920 10837 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.337383  9856 rpcz_store.cc:276] 0709 13:04:00.046358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.062950 (+ 16592us) service_pool.cc:224] Handling call
0709 13:04:01.336580 (+1273630us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b540051b654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:01.336724 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878286592}
I20260709 13:04:01.358696 10931 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.364354 10838 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:01.370172 11042 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:01.374274  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:01.383157  9194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:04:01.276422 10917 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
W20260709 13:04:01.401257  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 79295.2us
I20260709 13:04:01.396541  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:01.314201 11044 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:01.392082  9193 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:01.404791  9194 raft_consensus.cc:606] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Received request to transfer leadership
I20260709 13:04:01.339985  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:01.416824  9270 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:01.425577  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:01.472812  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 46633.1us
I20260709 13:04:01.475512  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:01.476040  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:01.478591 10931 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:01.490926  9268 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:01.474072  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:01.419435 11044 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:01.508292  9087 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:01.560415  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:01.475971 11040 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:01.483196 10997 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.497572  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:01.578398  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:01.574126  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 75451.9us
I20260709 13:04:01.589373  9270 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:01.560151 11027 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:01.468906 10838 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:01.604089  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 126974us
I20260709 13:04:01.603427 11027 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.599704  9263 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:01.666502  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:01.671821  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:01.672513 11030 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:01.583995 11040 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:01.684369 11049 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:04:01.686455  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:01.602560 10997 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.671273 11053 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:01.644747 11021 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:01.643940 11027 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:01.685039 10889 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:01.705421 10889 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.705839 10889 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:01.695276 11040 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:01.695822 11053 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.696821 10998 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:01.693691 11021 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:01.702427 11030 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:01.682044  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:01.717767  9191 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:01.732558 11025 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.756646 11025 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.757465 10997 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:01.797036  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:01.798038  9118 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:01.751199 11054 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:04:01.756063  9190 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:01.805827  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:01.733994 10998 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.806824  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:01.752452 10838 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:01.754312 11021 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:01.756104 11030 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:01.755473 11053 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:01.834683 11044 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:01.806996 11059 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:01.834761 10999 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.828504 11025 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:01.857664 11034 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:01.858248 11067 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:01.854584  9121 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:01.862589 11059 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.896319  9189 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:01.857156 11064 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:01.871464 10992 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.008s	sys 0.009s
I20260709 13:04:01.842577 10890 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:01.864005  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 303657us
I20260709 13:04:01.864714 11061 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:01.898391 10992 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.010s	sys 0.009s
I20260709 13:04:01.889983 10999 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.896378 11034 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:01.939967  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 304972us
I20260709 13:04:01.904327 10890 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.900372 11067 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:01.953260  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 471873us
I20260709 13:04:01.929270 11053 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:04:01.943861 11075 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.961509  9235 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:01.973740  9860 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:01.985996  9267 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:01.990682  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:01.990262  9269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.024s	sys 0.053s
W20260709 13:04:01.994088  9269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.025s	sys 0.053s
W20260709 13:04:02.001981 11071 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:02.003230 11026 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:02.003978 11026 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.004609 11059 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:02.006274 11026 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:02.008124 11067 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:01.981246 11040 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:02.009222 11040 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.009598 11040 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:01.913383 11061 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.017328  9235 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:01.992079 10889 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:02.024705 10999 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:01.957195 11030 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.024775  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:02.039610  9861 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:02.038947  9263 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:02.042033  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:02.036290 11059 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:02.036465 11067 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.042855  9266 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:02.042768 11059 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:01.976128 11075 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:02.045302 11055 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.568s	user 0.003s	sys 0.000s
I20260709 13:04:02.030442 11044 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:02.043200 11067 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.052614 10889 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:02.053385 10936 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:02.054174 11005 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 1.383s	user 0.010s	sys 0.001s
I20260709 13:04:02.036526 10998 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:02.064594  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1673) took 1944 ms. Trace:
I20260709 13:04:02.034173 11027 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:02.067266  9115 rpcz_store.cc:276] 0709 13:04:00.119795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.119960 (+   165us) service_pool.cc:224] Handling call
0709 13:04:00.120333 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:00.120695 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:00.120726 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:00.121174 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 13:04:00.221624 (+100450us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:00.221669 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:00.222168 (+   499us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:00.925622 (+703454us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:02.053789 (+1128167us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b2400168d88. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765347d9c4 00007f765347bacb 00007f7653636dc9 00007f7653636a55 00007f765363609d 00007f765626810e 00007f765627f88c 00007f765627ebb6 00007f765627e7d8 00007f7656288221
0709 13:04:02.064344 (+ 10555us) raft_consensus.cc:1685] finished
0709 13:04:02.064365 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.064497 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":985,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":34243712,"thread_start_us":100080,"threads_started":1,"wal-append.queue_time_us":471783}
I20260709 13:04:02.072124  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:02.073159 10838 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:02.075418 11034 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:02.077014  9116 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:02.074721 10838 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:02.077018 11059 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:02.098572  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 58112.3us
I20260709 13:04:02.102298  9861 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:02.102849  9269 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:02.105767  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:02.106315  9264 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:02.106529  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:02.107371  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:39752 (request call id 1767) took 1081 ms. Trace:
I20260709 13:04:02.108120  9265 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:02.080601 11075 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:02.125296 11027 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:02.126858  9264 rpcz_store.cc:276] 0709 13:04:01.025700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.029943 (+  4243us) service_pool.cc:224] Handling call
0709 13:04:02.104120 (+1074177us) spinlock_profiling.cc:243] Waited 85.1 ms on lock 0x7b54000ebad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566dba3d 00007f765b6abcd0 00007f7654c30254 00007f7654c301e2 00007f7654c30111 00007f7654c3008c 00007f7654c2f342 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:02.107266 (+  3146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195639040}
I20260709 13:04:02.109073  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:02.133023  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:02.123502  9162 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:02.134783  9123 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:04:02.135994 11075 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.136523 11075 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.137449  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:02.137904 11040 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:02.140579  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:02.141920 11040 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:04:02.110720  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 1807) took 1164 ms. Trace:
I20260709 13:04:02.109153  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:02.139150  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:02.150712  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:02.151443  9116 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:02.139611  9120 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:02.138676 11075 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:02.139824  9825 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:02.151907  9265 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:02.140097 10928 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:02.109769  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:02.140101  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:02.145606 10838 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:02.147395  9119 rpcz_store.cc:276] 0709 13:04:00.946219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.946355 (+   136us) service_pool.cc:224] Handling call
0709 13:04:02.110461 (+1164106us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54001905d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:02.110633 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341275904}
I20260709 13:04:02.161872  9122 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:02.161629  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:02.162348  9120 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:02.163220  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:02.165825  9265 raft_consensus.cc:2368] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3.
I20260709 13:04:02.166718  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:02.167590  9116 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:02.171329 11013 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.500s	user 0.008s	sys 0.026s
I20260709 13:04:02.173756  9119 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:02.179456  9235 leader_election.cc:400] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5.
I20260709 13:04:02.179855  9235 leader_election.cc:403] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:04:02.181000  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 72036.3us
I20260709 13:04:02.182158 10936 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5.
I20260709 13:04:02.182049 10909 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:02.183413 10936 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:02.184051 11069 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:02.184314 10936 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.184710 10936 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:02.184623 11069 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.161428 11053 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:04:02.160748 11075 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:02.189332 11075 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:02.189880 11075 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:02.197402  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:02.206033  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:02.198277  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:02.199033 10832 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:04:02.207762 10992 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:04:02.215049 11059 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:02.217150  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 309) took 1299 ms. Trace:
I20260709 13:04:02.217657  9192 rpcz_store.cc:276] 0709 13:04:00.917746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.917885 (+   139us) service_pool.cc:224] Handling call
0709 13:04:02.216901 (+1299016us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b540010c554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:02.217061 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388394880}
I20260709 13:04:02.218382  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:02.219508  9119 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:02.220608 11069 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:02.223295  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:02.212154  9263 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:02.238739  9114 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:02.222375 10890 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:02.245579  9116 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:02.245921  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:02.252641  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:02.208385  9265 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:02.248128  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:02.253247  9272 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:02.256515  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:02.257277  9235 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:02.257687  9162 leader_election.cc:400] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3.
I20260709 13:04:02.261020  9162 leader_election.cc:403] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:04:02.250939 11069 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:04:02.268673  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:43782 (request call id 1675) took 2080 ms. Trace:
I20260709 13:04:02.257719 10895 raft_consensus.cc:1064] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:04:02.273133  9114 rpcz_store.cc:276] 0709 13:04:00.167951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.168109 (+   158us) service_pool.cc:224] Handling call
0709 13:04:02.233351 (+2065242us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b540034ce54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566dba3d 00007f765b6abcd0 00007f7654c30254 00007f7654c301e2 00007f7654c30111 00007f7654c3008c 00007f7654c2f342 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:02.247845 (+ 14494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679590144}
I20260709 13:04:02.275656  9272 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:02.281078  9858 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:04:02.236122 11086 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:04:02.272651 10946 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:02.268030 10931 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:02.287575  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:02.289403  9861 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:02.275056 11019 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:04:02.287098 11006 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:02.295919 10931 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:02.322084  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:02.363186  9191 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:02.319373  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:02.318166  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:02.363348  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:02.306548  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:02.293772 11059 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:04:02.297120 10946 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:02.371840  9264 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:04:02.366451  9861 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:01.722105  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:02.385051  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:02.386114  9191 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:02.305909  9855 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:02.306003 10832 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:02.475703  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:02.457134  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:02.485176  9114 raft_consensus.cc:2368] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3.
I20260709 13:04:02.394464  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:02.313870 11084 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:02.486308  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:02.459770  9008 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 8a4bd074bf594b89b28b47d7482e8ac1 already in progress [suppressed 11 similar messages]
I20260709 13:04:02.511900  9122 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:02.438028 11023 raft_consensus.cc:1064] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:04:02.557106  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:02.558027  9191 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:02.558065  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:02.412546  9020 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:02.561374  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:02.462330 10989 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:02.429939 10991 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:02.466253  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:02.444551 10858 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:02.414363 11019 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.155s
I20260709 13:04:02.577284  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:02.495883 11024 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:02.572481  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:02.425982 10931 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.476145 10936 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.551240 11069 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.414336 10832 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:02.613876  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 310026us
I20260709 13:04:02.604041  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:02.439996 10946 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:02.573143 11084 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.578862 10989 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:02.580067 10991 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:02.647953 11046 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:02.629380 11030 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:02.610965  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 691) took 1227 ms. Trace:
I20260709 13:04:02.631897 11032 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:02.629995 11025 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:02.613420 11069 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.608012 10858 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.664315 11075 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.599210 11024 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:02.669018  9008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 209315us
I20260709 13:04:02.648669 11046 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.670219 11030 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:02.687247 11093 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:04:02.692515 11061 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:02.694635 10858 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:02.701543 11006 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3.
I20260709 13:04:02.752221 10920 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:04:02.833175  9194 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:02.834080 10917 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:02.835032 10917 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.838955 10917 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:02.686498 11025 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:02.666066  9856 rpcz_store.cc:276] 0709 13:04:01.372557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.383198 (+ 10641us) service_pool.cc:224] Handling call
0709 13:04:01.706696 (+323498us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:04:01.706895 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:01.755788 (+ 48893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.755818 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:01.755863 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:01.755895 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.582984 (+827089us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b54005a7154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:02.583076 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:02.594440 (+ 11364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141273216}
I20260709 13:04:02.789070 11085 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:02.685902 11032 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.812301 10936 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:02.704224 11030 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:02.970865  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:03.119428  9017 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42488 (request call id 237) took 3032 ms. Trace:
W20260709 13:04:03.522693  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.024s	sys 0.047s
W20260709 13:04:03.662046  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.025s	sys 0.052s
W20260709 13:04:03.580900  9164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.000s	sys 0.028s
W20260709 13:04:03.664407  9164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.001s	sys 0.029s
I20260709 13:04:03.273685 11068 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:02.853427  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:02.971130 10858 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:04:02.902884  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 320892us
I20260709 13:04:03.667203 10858 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:02.946430 11034 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:03.665905  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 1.04926e+06us
I20260709 13:04:03.668274 11034 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.412894 11067 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:03.669512 10858 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:03.400606 11099 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.005s	sys 0.018s
W20260709 13:04:03.670207 11099 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.005s	sys 0.019s
I20260709 13:04:03.666857  9857 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
W20260709 13:04:03.469018  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:02.892184 10838 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.671831 10838 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:03.669749  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:03.670921  9857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 581) took 2252 ms (client timeout 1972 ms). Trace:
I20260709 13:04:02.787606 10989 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:03.674113  9857 rpcz_store.cc:269] 0709 13:04:01.418006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.418159 (+   153us) service_pool.cc:224] Handling call
0709 13:04:02.853244 (+1435085us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:03.670779 (+817535us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15958400}
I20260709 13:04:03.046314 11069 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:03.666432 11068 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.677412 11068 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:03.679409  9195 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.020s	sys 0.024s
I20260709 13:04:03.120189  9020 catalog_manager.cc:5697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 2 to 3, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 3 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:03.086898 11046 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:03.240783 10895 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:03.658752  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 1.08639e+06us
W20260709 13:04:03.684538  9195 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.021s	sys 0.024s
W20260709 13:04:03.682605  9861 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.999s	user 0.009s	sys 0.022s
I20260709 13:04:03.684682  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 643) took 2101 ms. Trace:
I20260709 13:04:03.218119  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:03.660655  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:03.716257  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45924.2us
I20260709 13:04:02.788894 11024 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:03.716460  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48644us
I20260709 13:04:03.734577  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:02.905731 11097 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:04:03.732012  9862 rpcz_store.cc:276] 0709 13:04:01.560014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.560249 (+   235us) service_pool.cc:224] Handling call
0709 13:04:02.970678 (+1410429us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:03.660806 (+690128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5803904}
I20260709 13:04:03.716821  9089 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:03.668577 10865 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:03.740479  9861 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.013s	sys 0.023s
I20260709 13:04:03.670825 11034 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:03.755877  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:02.906363 10832 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:03.112177  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:03.762241  9270 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1497) took 1794 ms (client timeout 1921 ms). Trace:
I20260709 13:04:03.759435  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 591) took 1375 ms. Trace:
I20260709 13:04:03.440752 11086 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:03.763556  9270 rpcz_store.cc:269] 0709 13:04:01.975324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.200819 (+225495us) service_pool.cc:224] Handling call
0709 13:04:03.217924 (+1017105us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:03.754663 (+536739us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7106304}
W20260709 13:04:03.763648  9268 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.114s	user 0.012s	sys 0.038s
I20260709 13:04:02.892839 11021 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:03.785002  9268 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.136s	user 0.012s	sys 0.038s
I20260709 13:04:03.648373 10919 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:03.788143 10858 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:03.788527 10919 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.788863 10858 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.787245 11021 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.790669 10919 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:03.791024 11021 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:03.791172 10919 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:03.413080 11061 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:03.791636 11021 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:02.971553 10986 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:02.929610 10992 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:02.894526 10917 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:03.782729  9164 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:02.856134 11084 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:03.531663  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 422) took 1294 ms. Trace:
I20260709 13:04:03.787848 11086 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:03.763849  9861 rpcz_store.cc:276] 0709 13:04:02.383783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.675772 (+291989us) service_pool.cc:224] Handling call
0709 13:04:02.676131 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.676376 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:02.682825 (+  6449us) log.cc:844] Serialized 14 byte log entry
0709 13:04:03.759050 (+1076225us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:03.759098 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:03.759145 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.759220 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.759365 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17998,"prepare.queue_time_us":6267,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"thread_start_us":1057806,"threads_started":1}
I20260709 13:04:03.787992 11024 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:03.696480 11085 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:03.715895 10903 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:03.757107 10999 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:03.794381  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:03.762609 10936 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:03.762624 10998 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:03.783872 10931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.925s	user 0.018s	sys 0.040s
W20260709 13:04:03.831985  9826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.012s	sys 0.010s
W20260709 13:04:03.888600  9826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.012s	sys 0.010s
I20260709 13:04:03.794556 11067 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.801659  9266 rpcz_store.cc:276] 0709 13:04:02.195607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.197520 (+  1913us) service_pool.cc:224] Handling call
0709 13:04:02.451857 (+254337us) spinlock_profiling.cc:243] Waited 5.19 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:04:02.452144 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.698797 (+246653us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.701962 (+  3165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:02.751182 (+ 49220us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:02.810123 (+ 58941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.412938 (+602815us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54000eabd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:03.425839 (+ 12901us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.489970 (+ 64131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553505920}
I20260709 13:04:03.762311 11069 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:03.832319  9264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.009s	sys 0.012s
I20260709 13:04:03.808290 11075 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:03.897429  9264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.010s	sys 0.016s
I20260709 13:04:03.755404 11046 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:04:03.791072 10858 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:03.812700 11024 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:03.793669 11084 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:03.801990  9853 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:03.808921 11025 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:03.782032 10865 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:03.831836 10999 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.793390 10917 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.834015 11053 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:03.792143 11021 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:03.809903 10854 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:04:03.792064 11061 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.792364 10986 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:03.792721 10992 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.791723 10919 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:03.888243 10931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.018s	sys 0.040s
I20260709 13:04:03.881084 10832 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:03.956802 11027 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.048s	sys 0.116s
I20260709 13:04:03.957885 10989 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:03.957653  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:03.968487 10946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.000s	sys 0.022s
I20260709 13:04:03.926505 11084 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.967164 10986 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:03.982761 11027 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.049s	sys 0.116s
I20260709 13:04:03.935989 11069 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.831620 10903 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:03.925844  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 426) took 1565 ms. Trace:
I20260709 13:04:03.950075 10999 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:03.661434  9017 rpcz_store.cc:276] 0709 13:04:00.094732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.099910 (+  5178us) service_pool.cc:224] Handling call
0709 13:04:03.119278 (+3019368us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:00.504371 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:00.505028 (+   657us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:00.505085 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:04:00.505106 (+    21us) tablet.cc:662] Decoding operations
0709 13:04:00.505592 (+   486us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:00.505664 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:00.505699 (+    35us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:04:00.506044 (+   345us) tablet.cc:701] Row locks acquired
0709 13:04:00.506060 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:04:00.506212 (+   152us) write_op.cc:270] Start()
0709 13:04:00.506332 (+   120us) write_op.cc:276] Timestamp: P: 1783602240506166 usec, L: 0
0709 13:04:00.506351 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:04:00.506977 (+   626us) log.cc:844] Serialized 909 byte log entry
0709 13:04:00.549587 (+ 42610us) op_driver.cc:464] REPLICATION: finished
0709 13:04:00.550096 (+   509us) write_op.cc:301] APPLY: starting
0709 13:04:00.550172 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:00.553195 (+  3023us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:00.562090 (+  8895us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:00.625638 (+ 63548us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:00.651500 (+ 25862us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:04:00.657885 (+  6385us) write_op.cc:312] APPLY: finished
0709 13:04:00.658143 (+   258us) log.cc:844] Serialized 29 byte log entry
0709 13:04:00.658325 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:00.660413 (+  2088us) write_op.cc:454] Released schema lock
0709 13:04:00.661882 (+  1469us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:00.679111 (+ 17229us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:00.681332 (+  2221us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":250968,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":291,"apply.run_cpu_time_us":2708,"apply.run_wall_time_us":119825,"num_ops":2,"prepare.queue_time_us":12465,"prepare.run_cpu_time_us":2921,"prepare.run_wall_time_us":2923,"replication_time_us":42104}]]}
I20260709 13:04:03.892149 10936 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.926457 10991 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:03.902181 11024 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:03.987143 10919 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:03.886502 10998 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:03.967056  9853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 585) took 1709 ms (client timeout 1525 ms). Trace:
I20260709 13:04:03.987990 11112 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.977454 11068 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:03.979076 10898 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:04:03.991926 10946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.000s	sys 0.023s
W20260709 13:04:04.087256  9267 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.930s	user 0.003s	sys 0.038s
W20260709 13:04:04.207469  9267 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.050s	user 0.003s	sys 0.038s
I20260709 13:04:03.899447 11067 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.959807 11053 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:03.991797 11102 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:04.159247  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 420) took 1964 ms. Trace:
I20260709 13:04:04.212772  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1784) took 1723 ms. Trace:
W20260709 13:04:04.156612  9828 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.006s	sys 0.043s
I20260709 13:04:03.999665  9854 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:04:03.985193 11103 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:03.901119 11085 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:04.174923 11059 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.175530 11110 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:04.181667 11061 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:04.087319 11118 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:04.088156 11034 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:03.957902 11104 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:04.164134 11073 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:04.160276 11027 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:04.157619 10858 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:04.210421  9193 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.149s	user 0.007s	sys 0.035s
W20260709 13:04:04.214649  9828 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.006s	sys 0.043s
W20260709 13:04:04.220755  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 460703us
W20260709 13:04:04.221042  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 480501us
W20260709 13:04:04.228420  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 511693us
I20260709 13:04:04.185938 10998 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:04.222394 11032 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:04.213186  9267 rpcz_store.cc:276] 0709 13:04:02.488939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.489103 (+   164us) service_pool.cc:224] Handling call
0709 13:04:02.551934 (+ 62831us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.552162 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.552188 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.142820 (+590632us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b2c00112948. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:03.153922 (+ 11102us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.208118 (+1054196us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:04.208176 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.208269 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.208326 (+    57us) raft_consensus.cc:1685] finished
0709 13:04:04.208340 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.212709 (+  4369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6968064,"thread_start_us":1050813,"threads_started":1,"wal-append.queue_time_us":1018593}
I20260709 13:04:04.207136 10898 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:04.253547  9272 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:04.221717 10919 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:04.212227 11068 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:04.215112 11112 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:04.553982  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:04.477603  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:04.988020  9191 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:04.423789  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:04.646960  9197 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:04.436012  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:04.990440  9122 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:04.609974  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:04.992728  9161 leader_election.cc:400] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3.
I20260709 13:04:04.419909  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:04.540683  9020 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.022s	sys 0.039s
W20260709 13:04:04.995414  9020 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.206s	user 0.035s	sys 0.046s
I20260709 13:04:04.465632  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:04.508512  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:04.378479  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:04.223732 11103 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:05.343845  9236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.002s	sys 0.013s
W20260709 13:04:04.551890  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 307458us
W20260709 13:04:05.355870  9236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.003s	sys 0.014s
I20260709 13:04:05.009434  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 427) took 2549 ms. Trace:
I20260709 13:04:05.149080  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:04.905213  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1821) took 2648 ms. Trace:
I20260709 13:04:05.119477  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:05.058425  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 64246.3us
I20260709 13:04:04.620622  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:04.823396  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:04.243731 10946 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:04:04.997256  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 1.422e+06us
W20260709 13:04:05.362943  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 415473us
W20260709 13:04:05.140103  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 87906.1us
I20260709 13:04:05.373067  9189 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:04:04.240998 11099 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:04.994410  9193 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.933s	user 0.021s	sys 0.049s
W20260709 13:04:04.646646 11124 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
W20260709 13:04:04.647521 11111 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.483s	user 0.000s	sys 0.001s
W20260709 13:04:05.259837 10999 consensus_meta.cc:287] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.054s	user 0.008s	sys 0.004s
I20260709 13:04:04.241873 11119 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:05.358017  9119 rpcz_store.cc:276] 0709 13:04:02.256414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.256619 (+   205us) service_pool.cc:224] Handling call
0709 13:04:03.202859 (+946240us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b5800000c98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:04:03.203170 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:03.360541 (+157371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:03.360589 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.426131 (+ 65542us) log.cc:844] Serialized 26 byte log entry
0709 13:04:04.038312 (+612181us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:04.038351 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.038427 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.904806 (+866379us) raft_consensus.cc:1685] finished
0709 13:04:04.904832 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.905106 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":136315,"raft.queue_time_us":579292,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":9396,"spinlock_wait_cycles":4417920,"thread_start_us":486756,"threads_started":1,"wal-append.queue_time_us":883645}
I20260709 13:04:05.284924 11068 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:04.893637 11126 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
W20260709 13:04:04.517302 11117 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.312s	user 0.000s	sys 0.000s
I20260709 13:04:04.253955 10991 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.406261  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:04.184701  9853 rpcz_store.cc:269] 0709 13:04:02.252782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.257580 (+  4798us) service_pool.cc:224] Handling call
0709 13:04:03.111920 (+854340us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:03.956369 (+844449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6094208}
I20260709 13:04:04.828262 10936 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:05.406622  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 420205us
I20260709 13:04:04.606156 11123 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:05.408804  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:05.443539  9265 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:04.512220 10992 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:04.516533 11115 log.cc:927] Time spent T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.293s	user 0.000s	sys 0.000s
I20260709 13:04:05.410729  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:04.401062 11107 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.446738 11107 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:05.447109  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:05.010275 10898 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.448572  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:05.285198 10928 raft_consensus.cc:1027] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: attempt to promote peer 92919b70789041c8a0de0b3525154fd1: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:04:04.646665  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:05.452170  9196 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:05.452625  9196 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:04.253705 10889 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:05.113292 10919 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.356808  9268 rpcz_store.cc:276] 0709 13:04:02.438688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.545423 (+106735us) service_pool.cc:224] Handling call
0709 13:04:02.567278 (+ 21855us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.590392 (+ 23114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.590433 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:02.617246 (+ 26813us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.785421 (+1168175us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:03.785461 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.785541 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:04.987964 (+1202423us) raft_consensus.cc:1685] finished
0709 13:04:04.987991 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.988172 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7376,"raft.queue_time_us":23208,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":17099,"thread_start_us":1136179,"threads_started":1,"wal-append.queue_time_us":1335290}
I20260709 13:04:04.765491 10917 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:04.254771 10986 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:04.828521 11084 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:04.805311 11130 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:04.477728 11114 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.303s	user 0.000s	sys 0.000s
I20260709 13:04:05.237808 10903 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.112634 10989 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:04.345697 11086 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:05.010629 11021 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:05.249842 11075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.010s	sys 0.007s
I20260709 13:04:04.370662 11085 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:05.379544  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:05.119426 11024 consensus_meta.cc:287] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.900s	user 0.005s	sys 0.009s
I20260709 13:04:05.051924 11034 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.428256 10998 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:04.241716 11059 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.361924 11053 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:05.279191 10931 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:04.492317 10890 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:04.764485 11025 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.490741 10832 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:05.370838 10920 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:05.498333 11149 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:04:05.499050 10920 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:04.975431 11128 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:05.144575 11129 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:04.972921 11131 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:04.811839 11116 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:05.287164 11112 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.412350  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:05.122193 10858 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:04.518219 10865 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:05.143399 11027 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.459883  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1711) took 3990 ms. Trace:
I20260709 13:04:05.491876 11084 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:04.478802 11104 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:05.463392 11148 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:04:05.560568  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:04.242452 11102 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.468111  9268 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:05.474421  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 113207us
I20260709 13:04:05.408723 10946 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.453533  9196 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.489230 11024 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.416493 10999 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.470170 11069 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:04.427879 11118 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:05.479583 11075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.013s	sys 0.009s
I20260709 13:04:05.496011 11146 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:05.454412 10991 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.468848 11130 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.436641 11119 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.465646 10889 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.506145  9194 raft_consensus.cc:2368] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5.
I20260709 13:04:05.510978 11103 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.472816 10986 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.484344 10992 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:05.462209 11123 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.504264 10931 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:04.213040  9271 rpcz_store.cc:276] 0709 13:04:02.195099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.195209 (+   110us) service_pool.cc:224] Handling call
0709 13:04:02.216650 (+ 21441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.216870 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:03.678678 (+1461808us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b18003d3660. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f765628c4bc 00007f76535da48b 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f4455 00007f765a0dc3c7 00007f76566ee9a7 00007f76566fe0bf
0709 13:04:03.678878 (+   200us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.091423 (+412545us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.158783 (+ 67360us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:04.158863 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:04.158966 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:04.159141 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":148163,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":6709,"raft.queue_time_us":94400,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4790,"spinlock_wait_cycles":3470208,"thread_start_us":589175,"threads_started":2}
I20260709 13:04:05.484138 11068 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:05.488970 11086 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.475850 10989 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:04.164686  9263 rpcz_store.cc:276] 0709 13:04:02.360332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.362703 (+  2371us) service_pool.cc:224] Handling call
0709 13:04:02.368362 (+  5659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.368784 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.368818 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:02.369292 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.580454 (+211162us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:02.590011 (+  9557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:02.599490 (+  9479us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:03.912979 (+1313489us) raft_consensus.cc:1685] finished
0709 13:04:03.913027 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:03.925701 (+ 12674us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1314,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":6821,"thread_start_us":201764,"threads_started":1,"wal-append.queue_time_us":247395,"wal-append.run_cpu_time_us":36957,"wal-append.run_wall_time_us":2330325}
I20260709 13:04:05.549639  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 1794) took 1129 ms. Trace:
I20260709 13:04:05.550531  9193 rpcz_store.cc:276] 0709 13:04:01.464658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.464817 (+   159us) service_pool.cc:224] Handling call
0709 13:04:01.465184 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:01.482132 (+ 16948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:01.482182 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:02.201952 (+719770us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b18001c2ee0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:02.608834 (+406882us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b2c000fc1b8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:02.609352 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 13:04:02.996087 (+386735us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:05.405478 (+2409391us) log.cc:844] Serialized 14 byte log entry
0709 13:04:05.454542 (+ 49064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.454680 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.454721 (+    41us) raft_consensus.cc:1685] finished
0709 13:04:05.454735 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.454901 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":992693,"prepare.run_cpu_time_us":5566,"prepare.run_wall_time_us":36134,"raft.queue_time_us":8889,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":14202,"spinlock_wait_cycles":30457600,"thread_start_us":2778655,"threads_started":4,"wal-append.queue_time_us":1402236,"wal-append.run_cpu_time_us":3071,"wal-append.run_wall_time_us":1221068}
I20260709 13:04:05.497427 11053 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:05.514544 10890 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.557917 10858 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:05.565794  9196 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:05.457403 10893 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:05.570361  9119 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:05.575955  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:05.410437 11099 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.580811  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1783) took 3184 ms. Trace:
I20260709 13:04:05.661300  9272 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
W20260709 13:04:05.561021  9828 connection.cc:256] RPC call timeout handler was delayed by -1.3200027700000305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
I20260709 13:04:05.498315 10832 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.575819 11131 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.611503  9114 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:04:05.498281 11025 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:04:05.612320  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 170409us
I20260709 13:04:05.616137  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:05.616721  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:05.618381  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 59380.1us
I20260709 13:04:05.620739  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:05.581135 10865 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:05.673506  9116 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:04:05.673543  9118 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:05.642517  9198 raft_consensus.cc:2368] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5.
I20260709 13:04:05.583824 10917 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:05.648031  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 598) took 1694 ms. Trace:
I20260709 13:04:05.650386  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1787) took 1420 ms. Trace:
I20260709 13:04:05.676337  9270 rpcz_store.cc:276] 0709 13:04:04.230107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.230236 (+   129us) service_pool.cc:224] Handling call
0709 13:04:04.313922 (+ 83686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.558688 (+1244766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.558721 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.558764 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:05.558803 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.650212 (+ 91409us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540028d7d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:05.650256 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.650342 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2862449024}
I20260709 13:04:05.565564 11084 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.557766 11153 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.144s
I20260709 13:04:05.652514  9861 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:05.688086  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:05.653072 11159 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.662215 10920 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:05.653874 11157 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.653779  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38856
I20260709 13:04:05.654757  9191 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:05.656472  9267 rpcz_store.cc:276] 0709 13:04:04.420610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.371161 (+950551us) service_pool.cc:224] Handling call
0709 13:04:05.549549 (+178388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:05.627476 11107 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.656351  9117 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
W20260709 13:04:05.662475  9828 connection.cc:256] RPC call timeout handler was delayed by -1.1931027700000527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
I20260709 13:04:05.661341  9264 rpcz_store.cc:276] 0709 13:04:02.396688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.396883 (+   195us) service_pool.cc:224] Handling call
0709 13:04:02.487989 (+ 91106us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:02.561271 (+ 73282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:02.561300 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:03.025729 (+464429us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b2c00112948. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:03.026206 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:04:03.898007 (+871801us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:03.898047 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:03.898115 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:05.485401 (+1587286us) raft_consensus.cc:1685] finished
0709 13:04:05.485425 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.580739 (+ 95314us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53981,"raft.queue_time_us":55775,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":6221,"spinlock_wait_cycles":4258176,"thread_start_us":832609,"threads_started":1,"wal-append.queue_time_us":990640}
W20260709 13:04:05.702340  9828 connection.cc:256] RPC call timeout handler was delayed by -1.1319027700000106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
I20260709 13:04:05.674238 11160 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.666141  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:05.666669 11059 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.663596 11053 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.644138 11146 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.649879 10838 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.676211  9862 rpcz_store.cc:276] 0709 13:04:03.953645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:03.953789 (+   144us) service_pool.cc:224] Handling call
0709 13:04:03.954171 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:03.981878 (+ 27707us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:04.315546 (+333668us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.610258 (+294712us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.610334 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:04.610408 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.647808 (+1037400us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b54006013d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:05.647857 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.647978 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1026198,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":56,"raft.queue_time_us":30752,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":99673,"spinlock_wait_cycles":63106304,"thread_start_us":565188,"threads_started":2,"wal-append.queue_time_us":743219}
I20260709 13:04:05.682655  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1722) took 3124 ms. Trace:
I20260709 13:04:05.683822  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:05.683924  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:05.713920  9192 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:05.651407 11103 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:05.660771 11006 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.653563  9197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:05.554139 11129 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.551899 11116 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.716231  9197 raft_consensus.cc:606] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:05.702836  9828 connection.cc:256] RPC call timeout handler was delayed by -1.077902770000037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
I20260709 13:04:05.716625  9197 raft_consensus.cc:614] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:05.699782 11147 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:04:05.700227  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:05.675904 10865 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:05.656876 11068 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.730132  9860 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:05.707577 11146 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.659425 11086 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:05.648166 10992 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.717556  9828 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:05.687897 10917 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.664294  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:05.719832 11145 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:04:05.719460  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:05.708051  9195 rpcz_store.cc:276] 0709 13:04:02.557915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.579177 (+ 21262us) service_pool.cc:224] Handling call
0709 13:04:02.579499 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:02.688563 (+109064us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:03.730105 (+1041542us) log.cc:844] Serialized 14 byte log entry
0709 13:04:04.205335 (+475230us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:04.205415 (+    80us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:04.205489 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.682364 (+1476875us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54000a7154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:05.682423 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.682583 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20113,"prepare.queue_time_us":1292987,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":637,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":8072,"spinlock_wait_cycles":250227584,"thread_start_us":1437362,"threads_started":2,"wal-append.queue_time_us":781439,"wal-append.run_cpu_time_us":1252,"wal-append.run_wall_time_us":1002615}
I20260709 13:04:05.744814  9118 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:05.673488 10893 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.741707  9828 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:05.720012 11155 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 13:04:05.742723  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 78562.8us
I20260709 13:04:05.551221 11128 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.707598 11053 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.716356 11034 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:05.748353  9828 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:05.726114 10919 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:05.722834 10838 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:05.733364 10865 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:05.757426  9862 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:05.739203 10832 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.760924  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:05.719323 11006 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:05.722633 11103 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.709455 11159 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.742424 11160 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.722203 11023 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:05.714162 11157 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:06.326913  9194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 315) took 1861 ms (client timeout 1666 ms). Trace:
I20260709 13:04:05.762442  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:06.128957  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 84567.8us
W20260709 13:04:05.869827  9117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 1832) took 1620 ms (client timeout 2050 ms). Trace:
I20260709 13:04:05.750090 11146 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:06.210871  9087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.000s	sys 0.016s
I20260709 13:04:06.248965  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:05.874974  9115 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:06.563541  9087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.000s	sys 0.026s
W20260709 13:04:06.317482  9114 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 1834) took 1779 ms (client timeout 2130 ms). Trace:
W20260709 13:04:05.945021  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:06.394541  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 425415us
W20260709 13:04:06.564247  9114 rpcz_store.cc:269] 0709 13:04:04.537814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.538087 (+   273us) service_pool.cc:224] Handling call
0709 13:04:06.317291 (+1779204us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b540037f5d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:06.317409 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1423036672}
W20260709 13:04:06.254979  9198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 316) took 1774 ms (client timeout 1734 ms). Trace:
W20260709 13:04:06.193730  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 492006us
I20260709 13:04:06.432765  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:06.187529  9192 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:06.566514  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:06.392264  9196 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.731s	user 0.008s	sys 0.002s
I20260709 13:04:06.567850  9196 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:04:06.561561  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.210343  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:06.463238  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:06.569391  9858 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:04:06.313862  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:05.754878  9856 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:04:06.560222  9117 rpcz_store.cc:269] 0709 13:04:04.249193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.249313 (+   120us) service_pool.cc:224] Handling call
0709 13:04:05.869618 (+1620305us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400266e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:05.869747 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1520263936}
W20260709 13:04:06.558465  9194 rpcz_store.cc:269] 0709 13:04:04.465150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.465295 (+   145us) service_pool.cc:224] Handling call
0709 13:04:06.326400 (+1861105us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54000eef54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:06.326726 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1170960256}
W20260709 13:04:06.565583  9198 rpcz_store.cc:269] 0709 13:04:04.480255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.480403 (+   148us) service_pool.cc:224] Handling call
0709 13:04:06.213448 (+1733045us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54004245d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:06.254868 (+ 41420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595049728}
I20260709 13:04:06.564471  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:06.602537  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:05.761276 11034 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:04:05.822969 11152 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.257s	user 0.001s	sys 0.000s
I20260709 13:04:06.558094 11024 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:05.964787 11067 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.764038 10991 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.653425  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.664713  9264 raft_consensus.cc:2410] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:06.611831  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:06.625761  9189 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:06.626082  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:06.665362  9192 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
W20260709 13:04:06.392606 11138 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.000s	sys 0.010s
I20260709 13:04:05.865356  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1786) took 1636 ms. Trace:
I20260709 13:04:06.650215  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:06.328758 11068 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.762956 10999 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.180514 11123 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.627226  9861 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Advancing to term 5
I20260709 13:04:06.327847  9855 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:06.668040 11172 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:05.857755 11020 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:05.907956 10889 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:05.859488 11073 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:06.572240 10917 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:05.856308 11151 log.cc:927] Time spent T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 13:04:06.107882 11027 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.552615 10931 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:06.084470 11032 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:05.852082 10946 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.571764  9190 raft_consensus.cc:606] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:04:06.674024  9264 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:06.689630  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:06.328886 10992 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:06.571583 11021 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.693346  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:05.762403 11164 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.912s
I20260709 13:04:06.618397  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:06.606580 10998 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.572274 10893 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:05.759270 11149 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.903s
I20260709 13:04:06.089025 11102 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:06.210944 11118 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:06.753961  9195 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:06.697359  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.668957  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:06.756359  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:06.790768  9236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.169s	user 0.006s	sys 0.018s
I20260709 13:04:06.803619  9862 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:06.804078  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 325) took 1046 ms. Trace:
I20260709 13:04:06.806591  9195 rpcz_store.cc:276] 0709 13:04:05.758046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.746267 (+988221us) service_pool.cc:224] Handling call
0709 13:04:06.753758 (+  7491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.799514 (+ 45756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.804003 (+  4489us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.778332 11119 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:06.813628 10898 consensus_meta.cc:287] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.365s	user 0.008s	sys 0.005s
I20260709 13:04:06.814111 10898 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:06.815572  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.817075  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.818171  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.820535 11129 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:06.822656 11006 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:06.825316 10898 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.826424  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1790) took 2595 ms. Trace:
W20260709 13:04:06.830847 11085 consensus_meta.cc:287] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.345s	user 0.009s	sys 0.001s
I20260709 13:04:06.832013  9265 raft_consensus.cc:2410] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:06.831897  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.833563  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.834370  9268 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:04:06.834689  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 1803) took 1388 ms. Trace:
I20260709 13:04:06.835505  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.836694  9268 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:06.781415 11021 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:06.836971 11103 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.668831  9266 rpcz_store.cc:276] 0709 13:04:04.229485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.242272 (+ 12787us) service_pool.cc:224] Handling call
0709 13:04:04.603263 (+360991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.664101 (+1060838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.669685 (+  5584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:05.672993 (+  3308us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:05.685423 (+ 12430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.865023 (+179600us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003a1bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:05.865096 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:05.865275 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2410338048}
I20260709 13:04:06.837618  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.841370  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.841979  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.843681 11128 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:06.845638  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1728) took 1192 ms. Trace:
I20260709 13:04:06.719362  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:06.846266  9189 rpcz_store.cc:276] 0709 13:04:05.653620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.655028 (+  1408us) service_pool.cc:224] Handling call
0709 13:04:05.655228 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.665275 (+1010047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.845296 (+180021us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b54003b47d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:06.845525 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2217505664}
I20260709 13:04:06.572533 11084 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.617837 11116 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:05.857551 11139 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:06.690861 11138 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.004s	sys 0.013s
W20260709 13:04:06.860309  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 168849us
I20260709 13:04:06.193619 11025 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:06.776253  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 324) took 1018 ms. Trace:
I20260709 13:04:06.695377  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 604) took 1029 ms. Trace:
W20260709 13:04:06.768254 11169 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.625s	user 0.000s	sys 0.001s
I20260709 13:04:06.693253 11158 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:06.764955 11073 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:06.695600 11131 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:06.746387 11020 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:06.628755 11034 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:06.771607  9190 raft_consensus.cc:614] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:06.790423  9089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.047s	user 0.008s	sys 0.017s
W20260709 13:04:06.938613  9089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.198s	user 0.008s	sys 0.017s
W20260709 13:04:06.939018  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 1.2084e+06us
W20260709 13:04:06.791033  9121 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.017s	sys 0.007s
W20260709 13:04:06.939792  9121 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.222s	user 0.018s	sys 0.007s
W20260709 13:04:06.804541  9236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.183s	user 0.006s	sys 0.018s
I20260709 13:04:06.846762 11160 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:06.732456  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:06.849751 11159 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.859581 11021 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.952850 11157 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:06.953840  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 599) took 1338 ms. Trace:
I20260709 13:04:06.819469  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.956856  9271 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:04:06.871850  9192 rpcz_store.cc:276] 0709 13:04:05.757809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.689359 (+931550us) service_pool.cc:224] Handling call
0709 13:04:06.776172 (+ 86813us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.835222  9265 rpcz_store.cc:276] 0709 13:04:05.446497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.448351 (+  1854us) service_pool.cc:224] Handling call
0709 13:04:06.834422 (+1386071us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002a3354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:06.834612 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3179510528}
W20260709 13:04:06.875207  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 270302us
I20260709 13:04:06.876359  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:06.775672 11032 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:06.877853  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:06.826858  9269 rpcz_store.cc:276] 0709 13:04:04.231093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:04.231813 (+   720us) service_pool.cc:224] Handling call
0709 13:04:04.381058 (+149245us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:04.971896 (+590838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:04.971947 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:04.972576 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 13:04:05.411510 (+438934us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:05.411546 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:05.411615 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:06.826199 (+1414584us) raft_consensus.cc:1685] finished
0709 13:04:06.826222 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.826363 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":407860,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"thread_start_us":741581,"threads_started":2,"wal-append.queue_time_us":359935}
I20260709 13:04:06.728257  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:06.891108  9161 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:06.912556  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:06.944797  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:06.575008 11146 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:06.952863  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:06.962793  9087 connection.cc:256] RPC call timeout handler was delayed by -1.9174365769999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
W20260709 13:04:06.990454  9087 connection.cc:256] RPC call timeout handler was delayed by -1.6096365770000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
I20260709 13:04:06.989073  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:06.991289  9087 connection.cc:256] RPC call timeout handler was delayed by -1.4710365770000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
W20260709 13:04:06.989305  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 111548us
W20260709 13:04:06.991776  9087 connection.cc:256] RPC call timeout handler was delayed by -1.4485365770000271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:04:06.993155  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:06.993618  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:06.936180 10865 consensus_meta.cc:287] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.172s	user 0.008s	sys 0.004s
I20260709 13:04:06.993940  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:06.997133  9860 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:04:06.994391  9238 connection.cc:256] RPC call timeout handler was delayed by -1.7667581430000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260709 13:04:06.969961  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:06.999055 11141 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:07.002033 10931 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:04:06.960295  9853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.069s	sys 0.135s
I20260709 13:04:07.003719 11099 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:07.004160 11178 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:06.999992  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:06.999886  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:07.036917  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.037163  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:07.005569  9238 connection.cc:256] RPC call timeout handler was delayed by -1.7667581430000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 13:04:07.038925  9238 connection.cc:256] RPC call timeout handler was delayed by -1.6515581429999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1642s for a response
W20260709 13:04:07.039390  9238 connection.cc:256] RPC call timeout handler was delayed by -1.6515581429999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1642s for a response
W20260709 13:04:07.040126  9238 connection.cc:256] RPC call timeout handler was delayed by -1.3932581429999686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
I20260709 13:04:07.040063  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:07.040693  9238 connection.cc:256] RPC call timeout handler was delayed by -1.272658143000001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
I20260709 13:04:06.996578  9858 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:04:06.999281  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1729) took 1345 ms. Trace:
W20260709 13:04:07.037947  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 179406us
I20260709 13:04:07.006042  9265 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:06.936755 11020 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.010043  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:07.010826  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.048158  9857 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:06.875773  9856 rpcz_store.cc:276] 0709 13:04:05.661861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.664174 (+  2313us) service_pool.cc:224] Handling call
0709 13:04:05.704959 (+ 40785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.618739 (+913780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.691279 (+ 72540us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:06.943271 11034 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.002692 10917 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:06.931917 11069 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:07.014523 11172 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:07.043874  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.039381  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:06.931826 11139 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:06.932812 11158 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.004850 11146 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:07.016479  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 125467us
I20260709 13:04:07.054615  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:07.031241  9853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.070s	sys 0.136s
I20260709 13:04:07.004383 10865 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.041869  9191 rpcz_store.cc:276] 0709 13:04:05.653905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.698369 (+ 44464us) service_pool.cc:224] Handling call
0709 13:04:05.714713 (+ 16344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:06.629204 (+914491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.629249 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:06.629307 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:06.629359 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.999072 (+369713us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b5400417dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:06.999123 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.999233 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2102642560}
W20260709 13:04:07.164847  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 107897us
I20260709 13:04:07.083878  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.209352  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:07.029578 11046 raft_consensus.cc:1027] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: attempt to promote peer 92919b70789041c8a0de0b3525154fd1: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:04:07.058455  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:07.058276  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.215370  9265 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:07.219246  9857 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:07.035349 11159 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.068890  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.101897  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:07.226866  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.226965  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.227675  9856 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:07.227710  9116 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:07.228757  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.229359  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.229458  9856 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:04:07.230036  9116 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:07.230523  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.230927  9860 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:07.231055  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.231272  9856 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:07.067876  9114 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:07.243898  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.045411 11067 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:07.088339  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.045593 10946 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.245708  9122 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:07.084136  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:07.246804  9117 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:07.114382  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:07.246968  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:07.247619  9193 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
W20260709 13:04:07.182456  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 78946.8us
I20260709 13:04:07.113453  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:07.215504  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 157167us
I20260709 13:04:07.224491  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.248598  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:07.214398 11112 raft_consensus.cc:3055] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:07.112898  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:07.094419 11006 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:07.233722  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.057109  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.252058  9119 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:07.251506  9123 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:07.235944  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.249689  9192 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:07.235958  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.254135  9860 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:07.237574  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.250041  9195 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:07.255300  9857 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:07.256201  9267 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:07.226060  9858 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:07.112001 11183 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.220978 10917 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:07.239598  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 186255us
I20260709 13:04:07.239452  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:07.244772  9270 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:07.227867 10991 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:07.249603  9189 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:07.250618  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.068343 11068 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.204s
I20260709 13:04:07.095654  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.238440  9161 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:07.036439 11141 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.066995 11178 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.252888  9198 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:07.257699  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:07.382189  9089 connection.cc:256] RPC call timeout handler was delayed by -3.2416252840000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
I20260709 13:04:07.386384  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:07.389894  9089 connection.cc:256] RPC call timeout handler was delayed by -2.8233041199999889s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
I20260709 13:04:07.256821  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.307569  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.355038  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.275218  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.236781 11159 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:07.331110  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.393875  9191 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:04:06.963397  9854 rpcz_store.cc:276] 0709 13:04:05.615606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.620081 (+  4475us) service_pool.cc:224] Handling call
0709 13:04:05.658970 (+ 38889us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:05.693668 (+ 34698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:05.696059 (+  2391us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:05.749862 (+ 53803us) log.cc:844] Serialized 26 byte log entry
0709 13:04:06.916083 (+1166221us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:06.916141 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:06.916217 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:06.953583 (+ 37366us) raft_consensus.cc:1685] finished
0709 13:04:06.953615 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:06.953773 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3704,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6756,"thread_start_us":363038,"threads_started":1,"wal-append.queue_time_us":1154898}
W20260709 13:04:07.308832  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59629.1us
I20260709 13:04:07.397162  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:07.389895  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 151545us
I20260709 13:04:07.356030 11139 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.397643  9117 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:07.400085  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:07.394577  9860 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:07.403960  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.401901  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:07.348155 11187 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:04:07.410087  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:07.246011 11131 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:07.407011  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:07.271914 11006 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.381091 11021 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:07.250353 10889 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:07.352720 11176 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:04:07.406806  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.272362  9263 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:04:07.411921  9115 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:07.252077 11067 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:07.415342 10989 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.338862  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.391469 10991 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:07.491230 10991 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:07.251926 11112 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:07.496481 11112 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.282696 11181 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:07.497922 11181 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.274466 10946 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:07.400552  9266 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:07.382395 11185 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:07.426299  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.383982 11102 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:07.422542  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:07.408730  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:07.549795  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 141256us
W20260709 13:04:07.550060  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 127658us
I20260709 13:04:07.550781  9161 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:07.417585 11131 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:07.430647 11184 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:04:07.382798 11183 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.433928 11141 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.392689 11172 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:07.433744 11020 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.435072 10865 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.396778 11154 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:07.416134 11149 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:07.411571 11161 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:07.440868  9861 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:07.558327 11131 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:07.425807  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.640537  9122 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
W20260709 13:04:07.641842  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 89940.4us
I20260709 13:04:07.642127  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:07.445739  9196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1721) took 4888 ms (client timeout 2012 ms). Trace:
I20260709 13:04:07.354565 11158 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.405028 11159 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.457679  9114 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
W20260709 13:04:07.457784  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 50870.1us
I20260709 13:04:07.406035  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:07.490608 11178 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.498561  9115 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:07.501564  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.552366  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:07.459841 10998 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:04:07.499761 11073 consensus_meta.cc:287] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.000s
I20260709 13:04:07.488868  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.582095 11119 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:04:07.490633 11006 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.589324  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.711225  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.460965 11021 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.558748 10989 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:07.558633 11139 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:07.607563  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1707) took 1916 ms. Trace:
I20260709 13:04:07.396857 10917 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:07.643303  9196 rpcz_store.cc:269] 0709 13:04:02.557679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.557871 (+   192us) service_pool.cc:224] Handling call
0709 13:04:07.445374 (+4887503us) spinlock_profiling.cc:243] Waited 2.89 s on lock 0x7b5400399254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:07.445595 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6654638464}
I20260709 13:04:07.473829 10889 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.680059  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:07.698312  9198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 328) took 1899 ms (client timeout 2018 ms). Trace:
W20260709 13:04:07.723961  9198 rpcz_store.cc:269] 0709 13:04:05.799155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.617031 (+817876us) service_pool.cc:224] Handling call
0709 13:04:07.698071 (+1081040us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54003bd154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:07.698243 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033607424}
I20260709 13:04:07.581792  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.561026 11102 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:07.701670  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 295716us
I20260709 13:04:07.711354  9116 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:07.457823 11034 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.712733  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.692458 11158 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:07.560745 11185 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.721241  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.681077 11146 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:07.725988  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46038.4us
I20260709 13:04:07.717630  9121 rpcz_store.cc:276] 0709 13:04:05.691202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.691333 (+   131us) service_pool.cc:224] Handling call
0709 13:04:05.699405 (+  8072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:05.699591 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:06.940280 (+1240689us) log.cc:844] Serialized 14 byte log entry
0709 13:04:07.607058 (+666778us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:07.607133 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:07.607202 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.607294 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.607463 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49491,"prepare.queue_time_us":1121709,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":202474,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":141,"thread_start_us":1745980,"threads_started":3,"wal-append.queue_time_us":296164}
I20260709 13:04:07.735989  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:07.737368  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.737620  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:07.717676  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.714877 11085 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.683193 11067 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.731060 11164 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.740876 11020 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.741487 10890 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.741955 11185 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.733352  9162 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:07.765960  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 213683us
I20260709 13:04:07.767002  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.767838  9117 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:07.719882 10989 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:07.722110 11006 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:07.749219  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 78029.4us
I20260709 13:04:07.769061  9161 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:07.740559  9123 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.729483 11139 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:07.708536 11073 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:07.760110 11086 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:07.757274  9825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.684s (SENT) [suppressed 26 similar messages]
I20260709 13:04:07.748576 11085 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:07.756626 11193 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:07.755795 10931 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.757514  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 600) took 2116 ms. Trace:
I20260709 13:04:07.767999  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:07.777539  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.781812  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:07.762061 11164 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:07.787299  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.785836  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.759438 11183 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.794260  9122 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:07.770658 10989 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:07.771270 11006 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:07.806965  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.807821  9121 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:07.808102  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.811177 10889 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.783887 11193 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:07.814954 11193 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:07.799144 11164 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.783564 11085 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.779321 11181 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:07.837467  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:07.791378 10931 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:07.844131  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.795137 10890 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:07.847822  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:07.850601  9264 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:07.784999 11209 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:07.799885 11104 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:07.800004 11119 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:07.817900 11006 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:07.867512  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.869582  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.870364  9271 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:07.790453  9853 rpcz_store.cc:276] 0709 13:04:05.620717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.623624 (+  2907us) service_pool.cc:224] Handling call
0709 13:04:06.335519 (+711895us) spinlock_profiling.cc:243] Waited 7.66 ms on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:04:06.339794 (+  4275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:06.340289 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:06.340333 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:06.340960 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.241879 (+900919us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:07.248576 (+  6697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.281051 (+ 32475us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:07.711563 (+430512us) raft_consensus.cc:1685] finished
0709 13:04:07.716701 (+  5138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:07.737425 (+ 20724us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45471,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":17218,"spinlock_wait_cycles":17626496,"thread_start_us":869900,"threads_started":1,"wal-append.queue_time_us":704839}
I20260709 13:04:07.878027  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.882092  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:07.883021  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.871618  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.852655 10931 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.816944 10946 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:07.852159  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.849803  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:07.853468 10890 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:07.858868 11209 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.899793  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.901142  9198 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:07.901340  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:07.902200  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.851447 11210 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:04:07.858587  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:07.853232 10991 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:07.989030  9271 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:04:07.972208  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:07.992949  9271 pending_rounds.cc:85] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 1
I20260709 13:04:07.993369  9271 pending_rounds.cc:107] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:04:07.993743  9271 raft_consensus.cc:2889] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:07.994611  9858 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 2: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:07.984951  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.995379  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:07.998044  9264 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:07.996683 10890 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:08.002194 11073 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:08.000082  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.006287  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:07.996506 11209 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:08.000584 11206 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:04:07.905405  9856 raft_consensus.cc:2393] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 4: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:08.005196  9196 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:07.894891  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:08.010058  9117 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:04:08.011071  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.014688  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.018067  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.011771 11034 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:08.021032  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:07.998833 11102 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:08.019218 10991 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:08.028445  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:08.030272  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.029587  9117 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:08.033545  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.039038  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:08.039535  9089 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.040331  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:08.040859  9089 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:08.041531  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:08.042234  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 661) took 1049 ms. Trace:
I20260709 13:04:08.042744  9857 rpcz_store.cc:276] 0709 13:04:06.992758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.405308 (+412550us) service_pool.cc:224] Handling call
0709 13:04:07.405681 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:07.413316 (+  7635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:07.413349 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:07.413777 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:04:07.728841 (+315064us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:07.728913 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:07.729109 (+   196us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:08.042011 (+312902us) raft_consensus.cc:1685] finished
0709 13:04:08.042035 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:08.042174 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1131,"raft.queue_time_us":535,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":13249,"thread_start_us":174788,"threads_started":1,"wal-append.queue_time_us":298004}
I20260709 13:04:08.044600  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:08.049090  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.049243  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.050345  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.019933 11021 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:08.019546  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:08.054435  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 192301us
I20260709 13:04:08.056751  9193 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:08.067731  9189 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:04:08.030483 11215 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:08.087029  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:08.088634  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:07.908311 11139 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.097083  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:08.102526  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.103492  9193 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:08.086939 11153 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:08.086473 11102 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:04:08.131668  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 120629us
I20260709 13:04:08.133538  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.133644  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.136193 11130 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:08.166375  9123 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
W20260709 13:04:08.169147  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 151016us
W20260709 13:04:08.139310  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 98548us
I20260709 13:04:08.140950  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:08.173085  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:08.174036  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:08.174490  9857 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:08.174893  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:08.122305  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.138274 10890 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:08.179827  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.180979  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.148352 11102 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:08.170800  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:08.181926 10991 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.267418  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.228175  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.289553  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.265154  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:08.189263  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.252075  9116 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:08.366776  9265 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:08.226131  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.179023 11046 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:08.233889  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.232460  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:08.368780  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 79369.7us
I20260709 13:04:08.184166 11158 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:08.371820  9235 connection.cc:256] RPC call timeout handler was delayed by -1.1744020210000485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
W20260709 13:04:08.391171  9235 connection.cc:256] RPC call timeout handler was delayed by -1.0305264450000209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1975s for a response
W20260709 13:04:08.370111  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 144145us
I20260709 13:04:08.222570  9192 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:08.372037  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:08.367148  9198 raft_consensus.cc:2376] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4. Re-sending same reply.
I20260709 13:04:08.177278  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.188036 11023 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:08.405766 11181 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:08.406395 11181 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:08.406852 11181 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:08.382206  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:08.363669 10989 consensus_meta.cc:287] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.010s
I20260709 13:04:08.393250  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.412303  9825 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.264313  9855 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:08.368508 10991 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:08.377802 11216 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.389602 11210 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:08.397151 11158 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:08.295032 10890 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:08.404238  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:08.404912  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:08.405702  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.296023 11164 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.408628  9265 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:08.368134 10986 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:08.402446 10931 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.420261  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:08.444185  9119 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:08.284801 11006 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:08.376106  9856 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:04:08.447984  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:08.409958 11023 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:08.178535 11130 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.216s
I20260709 13:04:08.407967  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:38856 (request call id 94) took 2743 ms. Trace:
I20260709 13:04:08.426853 11210 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:08.446249  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 247722us
I20260709 13:04:08.461025  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.461019  9164 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:08.408401 11046 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:08.419440 10992 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:08.419162 10991 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:08.429582 11131 consensus_meta.cc:287] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.730s	user 0.008s	sys 0.039s
I20260709 13:04:08.455145  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.301723 11085 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.458803  9860 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:08.298447 11102 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:08.442538 11158 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:08.461676  9198 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:08.466238  9189 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:08.420341 10989 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:08.469223  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:08.472198  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.478673  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.459573 11216 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:08.467293 11193 consensus_meta.cc:287] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.002s
I20260709 13:04:08.480232  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:08.481694 11139 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:08.485265 11139 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:08.484781 11046 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.485690 11139 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:08.486164 11139 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:08.462992 11210 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:08.481750 11073 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:08.487620  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.488970  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.490130  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.490679  9266 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
W20260709 13:04:08.492004  9826 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.559s (SENT)
I20260709 13:04:08.495311  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.495491  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.495956 11006 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:08.496628  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:08.497437  9198 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:04:08.497100 11139 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.498231  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.499541 11139 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:08.501048  9270 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:08.470798 10992 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:08.504014  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.512472  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:08.513983  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 101771us
W20260709 13:04:08.494800 11159 consensus_meta.cc:287] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.021s
W20260709 13:04:08.530341  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 50201.3us
I20260709 13:04:08.531039  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.513072 10992 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:08.474675 11131 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:08.456189  9190 rpcz_store.cc:276] 0709 13:04:05.653043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:05.653574 (+   531us) service_pool.cc:224] Handling call
0709 13:04:08.370438 (+2716864us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b5400399254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566de853 00007f765b6ac60a 00007f7654c327d4 00007f7654c32762 00007f7654c32691 00007f7654c3260c 00007f7654c318c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:08.396784 (+ 26346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1983853440}
I20260709 13:04:08.501024 11193 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.524152 11215 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:08.525465  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.529587 10890 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.514483  9197 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:08.529172  9825 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.548810  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.491516  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:08.504608  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 132679us
W20260709 13:04:08.541543  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 74941.3us
I20260709 13:04:08.542529  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:08.482620 11156 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:08.544148  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:08.544327  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:08.546406 11237 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:08.529716 11216 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:04:08.538868 10917 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.810s	user 0.014s	sys 0.018s
I20260709 13:04:08.551679  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.554198  9195 raft_consensus.cc:2393] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:08.557036  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:08.584285  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.587301  9271 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
W20260709 13:04:08.556358  9826 leader_election.cc:336] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.725s (SENT)
I20260709 13:04:08.588811  9198 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:04:08.588522  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.589298  9198 pending_rounds.cc:85] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 4
I20260709 13:04:08.589850  9198 pending_rounds.cc:107] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.5
I20260709 13:04:08.590355  9198 raft_consensus.cc:2917] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Aborting config change with OpId 4.5: Aborted: Op aborted by new leader
I20260709 13:04:08.596503  9198 raft_consensus.cc:2889] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:08.597863  9825 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.534811 11159 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:08.614782  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 145628us
I20260709 13:04:08.617367  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:08.563346 11237 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:08.620651 11237 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:08.625702  9826 leader_election.cc:336] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.794s (SENT)
I20260709 13:04:08.625885  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.626188  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.526772 11023 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:08.630323  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45170us
I20260709 13:04:08.589495 11073 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:08.541390 11210 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:08.635684  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:08.636216 11085 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:08.588292  9266 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:04:08.599067 11215 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:08.599329  9190 raft_consensus.cc:606] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:08.612267 11216 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:08.653139 11069 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:08.602805 11084 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:08.592113 10917 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.597512 11232 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.667945  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.713857 11239 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:08.713670 11244 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
W20260709 13:04:08.716044  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 160347us
I20260709 13:04:08.716794  9164 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:08.653939 10931 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:08.647357 11073 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:08.755486  9859 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 2: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:08.600315 11231 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 13:04:08.667356 11215 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:08.710140 11232 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:08.710319 11084 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:08.763799 10992 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.764566 11156 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:08.769306 10917 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:08.769496 10992 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:08.752800 11085 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:04:08.754926 11230 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:04:08.746593 11181 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.772086 11085 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:08.753779 10989 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:08.752298 11216 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:08.759933 11164 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:08.787976  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:08.790722  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 187273us
I20260709 13:04:08.791177  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.656291 11235 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.787626 11085 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.822942  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:08.797844  9119 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:08.760309 11073 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:08.773120 11232 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.805181 11153 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:08.808182  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.839496  9264 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 1.
W20260709 13:04:08.812683  9269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 1816) took 1725 ms (client timeout 1885 ms). Trace:
W20260709 13:04:08.841017  9269 rpcz_store.cc:269] 0709 13:04:07.087437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.087621 (+   184us) service_pool.cc:224] Handling call
0709 13:04:08.791015 (+1703394us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:08.812596 (+ 21581us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3570048}
I20260709 13:04:08.773566 11215 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:08.720309 11241 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
W20260709 13:04:08.825368 11225 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 13:04:08.826086 11032 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:08.827040 11159 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:08.830149  9123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 301) took 1437 ms (client timeout 1559 ms). Trace:
W20260709 13:04:08.830965  9263 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 446) took 2016 ms (client timeout 1568 ms). Trace:
W20260709 13:04:08.880045  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 275516us
I20260709 13:04:08.880770  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:08.882637  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:08.845419  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.891783  9269 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:08.890650 11237 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.895126  9271 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:04:08.897150  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 271481us
I20260709 13:04:08.898504  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.851920 11181 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:08.903182  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:08.904191  9264 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:08.907191  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 148550us
I20260709 13:04:08.887032 11193 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:08.656849 11102 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.108s
I20260709 13:04:08.816953 11131 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:08.847618 11158 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.874585 11210 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:08.911069  9123 rpcz_store.cc:269] 0709 13:04:07.392897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.409832 (+ 16935us) service_pool.cc:224] Handling call
0709 13:04:08.829945 (+1420113us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b5400120a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:08.830082 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758073728}
W20260709 13:04:08.912714  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 277117us
I20260709 13:04:08.921612  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:08.933143  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.936117  9269 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:08.938257  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.939175  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:08.940251  9119 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:08.940676  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:08.941164  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:08.941511  9121 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:08.942431  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:08.944222  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:08.939405  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:08.947301  9122 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:08.953126  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 53201.4us
I20260709 13:04:08.956696  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:08.956987  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:08.957472  9862 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.958588  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:08.960955  9826 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:08.878723  9263 rpcz_store.cc:269] 0709 13:04:06.814549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.816774 (+  2225us) service_pool.cc:224] Handling call
0709 13:04:08.830769 (+2013995us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b5400062a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:08.830885 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":998064384}
I20260709 13:04:08.959811  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.974262  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.971576 11210 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:08.973639  9264 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:08.975643  9857 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:08.953271 11160 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:08.978384  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:08.945953 11084 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:08.989190 11084 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.976279 11203 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:08.999150  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:08.982795 11154 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:08.991160 10917 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.995579 10992 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:08.976477 10991 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:09.023804  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 82368.1us
I20260709 13:04:09.067631  9862 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
W20260709 13:04:09.070235  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 89224.7us
I20260709 13:04:09.071126  9161 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:08.978961 11245 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:09.076296  9087 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:09.019134 11209 consensus_meta.cc:287] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.001s	user 0.011s	sys 0.001s
I20260709 13:04:09.079769  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:09.083293  9087 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:09.093937  9087 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:09.012802  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:09.021775 10917 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:09.112618  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:09.133630  9264 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:09.134783  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1527) took 1144 ms. Trace:
I20260709 13:04:09.135092  9264 rpcz_store.cc:276] 0709 13:04:07.990787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.117759 (+1126972us) service_pool.cc:224] Handling call
0709 13:04:09.133404 (+ 15645us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.134114 (+   710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.134725 (+   611us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.081972 11203 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:09.248440 11023 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:09.250456 11085 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.251173  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1831) took 1252 ms. Trace:
I20260709 13:04:09.250926 11023 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.251535  9264 rpcz_store.cc:276] 0709 13:04:07.999094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.137194 (+1138100us) service_pool.cc:224] Handling call
0709 13:04:09.250715 (+113521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.250877 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.250903 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.250943 (+    40us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:09.250973 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.251029 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.251125 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:09.111198 11158 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.111898 11102 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:09.112339 11021 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:09.085697 11181 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:04:09.256489  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 429) took 2690 ms. Trace:
W20260709 13:04:09.258319  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 308619us
I20260709 13:04:09.252679 11085 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.117770 10917 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:09.111353 11247 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:09.287375  9267 rpcz_store.cc:276] 0709 13:04:06.566264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:06.663554 (+ 97290us) service_pool.cc:224] Handling call
0709 13:04:06.696927 (+ 33373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:07.256702 (+559775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.256161 (+1999459us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54000fa5d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:09.256410 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275029760}
I20260709 13:04:09.315598  9267 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:09.316449  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 455) took 1159 ms. Trace:
I20260709 13:04:09.112084 11209 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.316777  9267 rpcz_store.cc:276] 0709 13:04:08.157336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.296773 (+1139437us) service_pool.cc:224] Handling call
0709 13:04:09.315410 (+ 18637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.316175 (+   765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.316389 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:09.328238  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.260756 11203 raft_consensus.cc:697] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:09.273048 11021 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:09.297227 11231 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:09.346997  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1726) took 1565 ms. Trace:
I20260709 13:04:09.344745 11203 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.366626  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 460) took 1207 ms. Trace:
I20260709 13:04:09.388118  9267 rpcz_store.cc:276] 0709 13:04:08.159160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.317660 (+1158500us) service_pool.cc:224] Handling call
0709 13:04:09.365928 (+ 48268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.366173 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.366211 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.366268 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:09.366313 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.366384 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.366546 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:09.323307 11258 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:04:09.405031  9193 raft_consensus.cc:2368] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4.
I20260709 13:04:09.356360  9116 rpcz_store.cc:276] 0709 13:04:07.781768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.782036 (+   268us) service_pool.cc:224] Handling call
0709 13:04:07.784367 (+  2331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.369278 (+584911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.346759 (+977481us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b540034ce54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:09.346938 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1075221376}
W20260709 13:04:09.436481  9115 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 1713) took 2388 ms (client timeout 1778 ms). Trace:
W20260709 13:04:09.437094  9115 rpcz_store.cc:269] 0709 13:04:07.036811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.036939 (+   128us) service_pool.cc:224] Handling call
0709 13:04:09.424522 (+2387583us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b5400285354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:09.424694 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860619520}
I20260709 13:04:09.354934 11253 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.181s
I20260709 13:04:09.353564 11021 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:09.392529 11231 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:09.505471  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1832) took 1332 ms. Trace:
I20260709 13:04:09.507704  9267 rpcz_store.cc:276] 0709 13:04:08.173280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.482415 (+1309135us) service_pool.cc:224] Handling call
0709 13:04:09.504651 (+ 22236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.505014 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.505052 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.505119 (+    67us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:09.505164 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.505237 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.505389 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:09.495241  9853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.068s	sys 0.084s
I20260709 13:04:09.457404 11023 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.514863 11231 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:09.534085  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 716) took 2118 ms. Trace:
I20260709 13:04:09.545043  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1830) took 1633 ms. Trace:
I20260709 13:04:09.545676  9269 rpcz_store.cc:276] 0709 13:04:07.911490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.936886 (+1025396us) service_pool.cc:224] Handling call
0709 13:04:09.543907 (+607021us) spinlock_profiling.cc:243] Waited 4.85 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:04:09.544211 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.544536 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.544579 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.544663 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:09.544721 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.544808 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.544958 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11148672}
I20260709 13:04:09.534807  9860 rpcz_store.cc:276] 0709 13:04:07.415484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.997065 (+581581us) service_pool.cc:224] Handling call
0709 13:04:07.997431 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.475311 (+477880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.533705 (+1058394us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54004538d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:09.533989 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060677504}
W20260709 13:04:09.522081  9853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.070s	sys 0.085s
I20260709 13:04:09.569237  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:09.570479  9861 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:09.571012  9861 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 717) took 2155 ms (client timeout 2865 ms). Trace:
W20260709 13:04:09.571506  9861 rpcz_store.cc:269] 0709 13:04:07.415846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.682779 (+266933us) service_pool.cc:224] Handling call
0709 13:04:09.568809 (+1886030us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:09.570919 (+  2110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729344}
I20260709 13:04:09.574801  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 454) took 1417 ms. Trace:
I20260709 13:04:09.575250  9264 rpcz_store.cc:276] 0709 13:04:08.157048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.252465 (+1095417us) service_pool.cc:224] Handling call
0709 13:04:09.252771 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.252961 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:09.315247 (+ 62286us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.574481 (+259234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.574532 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:09.574578 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.574640 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.574742 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5963,"prepare.queue_time_us":99447,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":11142,"thread_start_us":276936,"threads_started":2}
I20260709 13:04:09.585561  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.586277  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:09.585767  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.588251  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.589641  9272 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:09.585457  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1749) took 1379 ms. Trace:
W20260709 13:04:09.591404  9826 leader_election.cc:336] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.759s (SENT)
I20260709 13:04:09.591888  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:09.591053  9198 rpcz_store.cc:276] 0709 13:04:08.206382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.598045 (+391663us) service_pool.cc:224] Handling call
0709 13:04:09.584760 (+986715us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b5800003098. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:04:09.584980 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.585160 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:09.585185 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.585224 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:09.585254 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.585309 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.585406 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11031296}
I20260709 13:04:09.593415  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:09.588515 11264 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:04:09.596676  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:09.600894 10928 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:09.573125 11158 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:04:09.596809  9264 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:09.595937 11116 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:09.596212 11231 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.582931  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 453) took 1425 ms. Trace:
I20260709 13:04:09.602066  9162 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:09.595621  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:09.592458 10991 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:09.602102 11252 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.491s	user 0.003s	sys 0.000s
I20260709 13:04:09.610600  9198 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:09.612353  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:09.616725  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:09.604846  9263 rpcz_store.cc:276] 0709 13:04:08.156749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.250831 (+1094082us) service_pool.cc:224] Handling call
0709 13:04:09.251143 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:09.251342 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:09.251657 (+   315us) log.cc:844] Serialized 14 byte log entry
0709 13:04:09.524795 (+273138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:09.567015 (+ 42220us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:09.569342 (+  2327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:09.572529 (+  3187us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:09.581848 (+  9319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41472,"prepare.queue_time_us":42944,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":21584,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":6459,"thread_start_us":242784,"threads_started":1,"wal-append.queue_time_us":342816}
I20260709 13:04:09.617879  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:09.618666  9198 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:09.619810 11073 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.768s	user 0.012s	sys 0.000s
I20260709 13:04:09.614981 10928 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:09.609450 11158 consensus_queue.cc:1243] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:04:09.623984  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 511458us
I20260709 13:04:09.625568  9087 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:09.616938 11023 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:09.627101  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:09.620296 11073 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.626139  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:09.629052  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:09.620888 11209 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:09.632718  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:09.633906  9235 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:09.635906 11251 consensus_queue.cc:1059] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:09.636552  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:09.636981 11251 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:09.637475 11251 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:09.637892 11251 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:09.638563 11253 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:09.639411 11253 raft_consensus.cc:3055] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:09.639909 11253 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:09.641543  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:09.642220  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:09.629158 11247 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:09.649546  9265 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:09.631105 11128 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:09.641425 11158 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:09.644573 11023 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:09.647945 11253 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.653424 11128 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:09.629541 11268 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:09.656591 11006 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 13:04:09.693432  9197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1730) took 1679 ms (client timeout 1874 ms). Trace:
W20260709 13:04:09.690299  9859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 616) took 2775 ms (client timeout 2062 ms). Trace:
W20260709 13:04:09.652814 11154 consensus_meta.cc:287] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.000s
W20260709 13:04:09.696125  9197 rpcz_store.cc:269] 0709 13:04:08.014249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.019971 (+  5722us) service_pool.cc:224] Handling call
0709 13:04:09.693198 (+1673227us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54000a7654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:09.693359 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105888000}
W20260709 13:04:09.696446  9859 rpcz_store.cc:269] 0709 13:04:06.914708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.224201 (+309493us) service_pool.cc:224] Handling call
0709 13:04:09.690075 (+2465874us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54005a7154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:09.690219 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3458789632}
I20260709 13:04:09.666734 11073 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.705349 11154 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:09.667441 11253 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:09.676950 11215 consensus_meta.cc:287] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.798s	user 0.008s	sys 0.003s
W20260709 13:04:09.707785 11084 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.718s	user 0.008s	sys 0.003s
I20260709 13:04:09.707542 11215 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.669528 11067 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:09.710706  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.711905  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.713142  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.664711 11023 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.714913  9263 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:09.666991 11143 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:09.716163  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.716158 11143 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:09.716874  9269 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:09.717512 11143 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.718057  9269 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:09.718719 11143 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:09.695648 11268 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:09.705031 11236 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.720630 11236 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.721139 11215 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:09.723327  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:09.696856 11128 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:09.648993 11249 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:09.713300  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.740648  9272 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:09.741408 10928 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:09.741549  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.742445  9272 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:09.743264  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:09.726155  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:09.744168  9193 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1745) took 1589 ms (client timeout 1655 ms). Trace:
I20260709 13:04:09.744817  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:09.726848 11232 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.875s	user 0.004s	sys 0.007s
I20260709 13:04:09.732178 11128 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.733517  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:09.733520 11161 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:09.736131  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:09.736871  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.738099 11112 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:09.720072 11268 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:09.744560  9193 rpcz_store.cc:269] 0709 13:04:08.155119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.189076 (+ 33957us) service_pool.cc:224] Handling call
0709 13:04:09.743998 (+1554922us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000a6754. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:09.744116 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2356194560}
I20260709 13:04:09.768594  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:09.653749 11231 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.750953  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.752099  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.756691  9860 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:09.773047  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:09.669555 10992 consensus_meta.cc:287] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.599s	user 0.003s	sys 0.007s
I20260709 13:04:09.766381  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:09.774058  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:09.774395  9861 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:09.764430 11247 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:09.774749  9115 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:09.775139 11247 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.776177  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.776765  9264 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:09.777408 11247 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:09.777534  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.778148  9264 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:09.771688  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:09.768631  9269 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:09.770028  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:09.764269 11232 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.770699  9263 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:09.764561 11141 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:09.747491  9859 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:09.778209  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:09.783933  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:09.781817 11270 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:09.785971  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.782171  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.668883 10917 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.794224  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:09.784228  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:09.808214  9119 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:09.783241  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:09.809218  9115 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:09.802711  9856 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:09.766835 11193 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:04:09.810575 11193 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:09.810928 11193 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:09.781888 11032 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:04:09.778654 11119 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:04:09.802798 11141 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.812808 11119 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:09.811823 10865 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:04:09.813299 11119 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:09.805150 11139 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:04:09.814183 11119 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.815155 11139 raft_consensus.cc:3055] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:09.784966  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:09.784586  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:09.797108  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:09.798434  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.799777  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:09.803391  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.805792  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:09.811813 11193 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.766943 11215 consensus_queue.cc:1059] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:04:09.815795  9193 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:09.787029 11249 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:09.817564  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.808631  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:09.786608  9192 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1737) took 1737 ms (client timeout 1508 ms). Trace:
I20260709 13:04:09.782325 10992 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:09.833372  9192 rpcz_store.cc:269] 0709 13:04:08.049493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.050143 (+   650us) service_pool.cc:224] Handling call
0709 13:04:09.786381 (+1736238us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54000a6c54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:09.786533 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394358144}
I20260709 13:04:09.833940  9235 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:09.829229  9197 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:09.834594  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:09.823714 11139 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:09.782646  9861 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:09.838970  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.806602 11153 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:09.845337  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:09.778817 11161 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.846918  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:09.849737  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:09.853094  9192 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:09.856124  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.850817  9193 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:09.858579  9269 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:09.858579  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:09.834969 11119 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:09.846637 11084 raft_consensus.cc:1064] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:04:09.816342 11102 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:04:09.830750 10865 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:09.801424 11231 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:04:09.844363 11085 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:09.845185 11249 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:09.855723 11069 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:09.859529 11023 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.860598 10992 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:09.816884 11270 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.859983 11210 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:09.864372 11193 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:09.880592  9264 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:04:09.869762 11084 raft_consensus.cc:1068] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Unable to promote non-voter 92919b70789041c8a0de0b3525154fd1: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:04:09.884200  9853 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:04:09.866737 10865 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:09.850961 11021 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.846591 11251 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.890605 11069 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:09.890769 10946 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:09.891953  9859 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:09.893422 11128 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:09.894446 11270 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:09.894745  9862 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:09.873515 11139 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.895481 11102 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:09.895370  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:09.896122  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:09.897665  9861 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:09.896708 10865 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:09.907361  9858 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:09.891602  9855 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:09.904770  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:09.896329 11237 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.908200  9859 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:09.856518 11153 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.889379  9853 pending_rounds.cc:85] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 5
I20260709 13:04:09.912696  9853 pending_rounds.cc:107] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:04:09.903815 11069 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.913007  9853 raft_consensus.cc:2889] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:09.913775 11153 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.902132 10946 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:09.898958 11021 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.921361  9856 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:09.922639  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:09.906847 11102 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:09.919679 10865 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.924680 11102 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.887100 11023 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.929723  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:09.915670 11069 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:09.914789 11247 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:09.931064 11247 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:09.931381 11251 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.932216 11023 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:09.931910 11247 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.915930 10917 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:09.914691 10931 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.916900 11249 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.917055 10946 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.933135 10931 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.933996 10946 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:09.918162 11232 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:09.921407 11268 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.933117 11247 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:09.918360 11032 raft_consensus.cc:3055] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:09.937213 11032 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:09.938926 11032 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:09.940173 11032 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:09.946023 10931 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:09.929343 11161 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:09.915151 11146 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:09.975397 11268 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:10.010097  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.021070  9194 raft_consensus.cc:2410] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:04:09.916397  9855 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:09.918776  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.036835  9857 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:09.928303  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.038168  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:09.912055  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:09.928573  9860 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:09.930500  9861 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 2: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:10.019086  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:09.925741  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.049345  9862 raft_consensus.cc:1813] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Handling vote request from an unknown peer ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.050259  9862 raft_consensus.cc:2410] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:04:10.051733  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:10.054215  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.055055  9197 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:10.056536  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:10.057344  9269 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:10.039357  9858 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:09.927809  9856 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:10.011458  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.927563 10865 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:10.105372 10890 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.926760 11102 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:10.106479  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:09.926625 11139 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:10.147403 10890 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:09.889614 11210 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.060504  9855 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEARNER]: Advancing to term 4
I20260709 13:04:10.126878  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:10.129025 11211 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:10.132211  9235 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:10.133265  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 100376us
I20260709 13:04:10.128131  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:10.143347  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.146035  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:10.148576  9857 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:10.161193  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:10.162860  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 131462us
I20260709 13:04:10.169807  9197 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:10.172899  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:10.175136 11268 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:10.175448  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.175809 11268 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:10.175835 11210 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:10.176523 11268 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:10.176724  9197 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:10.177881  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:10.179276  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.169836  9235 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:10.181815  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:10.186367 11256 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.170589 11211 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:10.191149 11118 raft_consensus.cc:3055] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:10.191617 11118 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:10.191890  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.192257  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.193110  9856 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 2: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:10.193333  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.194093  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.195274  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.196833  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:10.198046  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.199630 11210 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:10.200205 11210 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:10.200640 11210 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:10.200865 11256 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:10.209077  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:10.203999 10890 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:10.205029  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 451) took 2048 ms. Trace:
W20260709 13:04:10.212756  9828 leader_election.cc:336] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.666s (SENT)
I20260709 13:04:10.212670  9265 rpcz_store.cc:276] 0709 13:04:08.156159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.893520 (+737361us) service_pool.cc:224] Handling call
0709 13:04:10.204583 (+1311063us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b54001c5cd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:10.204794 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722666240}
I20260709 13:04:10.213378  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.215013 11193 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:10.194191  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1828) took 2416 ms. Trace:
W20260709 13:04:10.216634  9195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1734) took 2168 ms (client timeout 2053 ms). Trace:
I20260709 13:04:10.205363  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:10.216657  9270 rpcz_store.cc:276] 0709 13:04:07.777812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.778032 (+   220us) service_pool.cc:224] Handling call
0709 13:04:07.790902 (+ 12870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.501469 (+710567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.193682 (+1692213us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54004439d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:10.194009 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632903936}
I20260709 13:04:10.217758  9116 raft_consensus.cc:2376] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5. Re-sending same reply.
W20260709 13:04:10.218992  9195 rpcz_store.cc:269] 0709 13:04:08.048491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.048815 (+   324us) service_pool.cc:224] Handling call
0709 13:04:10.216074 (+2167259us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54000a7154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:10.216372 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159582080}
W20260709 13:04:10.220096  9828 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.734s (SENT)
I20260709 13:04:10.206997  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:10.220754  9828 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.221146  9122 raft_consensus.cc:2376] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5. Re-sending same reply.
I20260709 13:04:10.221740 11193 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:10.193235 11118 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.216732  9189 raft_consensus.cc:1240] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:10.206086  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:10.201630  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.217691  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:10.226830  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.228298  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:10.235406  9195 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:10.238137  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.237208 11118 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:10.241641  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.242506  9857 raft_consensus.cc:1813] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Handling vote request from an unknown peer ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.243019  9857 raft_consensus.cc:2410] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:04:10.240414  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1753) took 1981 ms. Trace:
I20260709 13:04:10.245721  9195 rpcz_store.cc:276] 0709 13:04:08.257416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.220327 (+1962911us) service_pool.cc:224] Handling call
0709 13:04:10.226645 (+  6318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.237464 (+ 10819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.238377 (+   913us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.246634  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:10.248152  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52079.3us
W20260709 13:04:10.250079  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.252296  9858 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 2: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:10.253875  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.255095  9853 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:10.255020  9197 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:10.255928  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 110348us
W20260709 13:04:10.260412  9194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1721) took 2380 ms (client timeout 1850 ms). Trace:
W20260709 13:04:10.261022  9194 rpcz_store.cc:269] 0709 13:04:07.880314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.881805 (+  1491us) service_pool.cc:224] Handling call
0709 13:04:10.260067 (+2378262us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b5400398854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:10.260320 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4915206912}
I20260709 13:04:10.256470  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1754) took 1997 ms. Trace:
I20260709 13:04:10.268714  9197 rpcz_store.cc:276] 0709 13:04:08.259282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.224594 (+1965312us) service_pool.cc:224] Handling call
0709 13:04:10.239076 (+ 14482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.256213 (+ 17137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.256421 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:10.275005  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.275137  9271 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 450) took 2118 ms (client timeout 1794 ms). Trace:
I20260709 13:04:10.274554  9191 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:10.244377  9192 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:10.251168 11210 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:10.275714  9271 rpcz_store.cc:269] 0709 13:04:08.155817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.645659 (+489842us) service_pool.cc:224] Handling call
0709 13:04:10.271847 (+1626188us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54000fafd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:10.274262 (+  2415us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423436544}
I20260709 13:04:10.279100  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1723) took 2398 ms. Trace:
I20260709 13:04:10.279616  9196 rpcz_store.cc:276] 0709 13:04:07.881011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:07.882901 (+  1890us) service_pool.cc:224] Handling call
0709 13:04:07.883237 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:08.009821 (+126584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.278567 (+2268746us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b540010ca54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:10.278903 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280015104}
I20260709 13:04:10.281879  9196 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:10.282655  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1759) took 1820 ms. Trace:
I20260709 13:04:10.283680  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:10.283056  9196 rpcz_store.cc:276] 0709 13:04:08.461956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.280841 (+1818885us) service_pool.cc:224] Handling call
0709 13:04:10.281475 (+   634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.282370 (+   895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.282566 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.285779  9266 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:10.290100  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 348) took 1307 ms. Trace:
I20260709 13:04:10.288988 11193 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.292215  9854 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:10.295797  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:10.300604  9190 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:10.302053 11260 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:10.303020  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.303793  9193 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:10.306288  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1756) took 1924 ms. Trace:
I20260709 13:04:10.327418  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:10.328707  9194 rpcz_store.cc:276] 0709 13:04:08.381607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.305240 (+1923633us) service_pool.cc:224] Handling call
0709 13:04:10.305719 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.305944 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.305979 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:10.306026 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:10.306043 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.306099 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.306190 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.309257  9087 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:10.309965 11141 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:10.311367 11193 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.331861  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:10.332532  9194 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:10.333055 11220 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:10.333343  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1745) took 1779 ms. Trace:
I20260709 13:04:10.314093  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1751) took 2086 ms. Trace:
I20260709 13:04:10.321235  9020 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36634 (request call id 338) took 10162 ms. Trace:
I20260709 13:04:10.315716 11210 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:10.335112  9828 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:10.317788  9018 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36610 (request call id 323) took 8187 ms. Trace:
I20260709 13:04:10.326617  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:10.334010  9189 rpcz_store.cc:276] 0709 13:04:08.227241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.835218 (+607977us) service_pool.cc:224] Handling call
0709 13:04:10.216345 (+1381127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.223773 (+  7428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.313852 (+ 90079us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54000a7150. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:10.314012 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3175438208}
I20260709 13:04:10.336336 11220 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.324291  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:10.315590  9263 consensus_meta.cc:287] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.510s	user 0.013s	sys 0.007s
I20260709 13:04:10.338346  9263 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:10.338706  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:10.339900  9272 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:10.325783  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1750) took 2102 ms. Trace:
I20260709 13:04:10.324494 11260 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.341879  9190 rpcz_store.cc:276] 0709 13:04:08.222954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.774241 (+551287us) service_pool.cc:224] Handling call
0709 13:04:09.786677 (+1012436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.324137 (+537460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.324974 (+   837us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54000a6c54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:10.325550 (+   576us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54000a6c50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:10.325710 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3454814336}
I20260709 13:04:10.343946  9190 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:04:10.344337  9190 pending_rounds.cc:85] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 4
I20260709 13:04:10.344583  9190 pending_rounds.cc:107] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.5
I20260709 13:04:10.344645 11260 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:10.345072  9190 raft_consensus.cc:2917] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted by new leader
I20260709 13:04:10.345642  9190 raft_consensus.cc:2889] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:10.347200  9855 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:10.333454 11193 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:10.313278  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1829) took 2405 ms. Trace:
W20260709 13:04:10.316259  9198 consensus_meta.cc:287] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.697s	user 0.005s	sys 0.014s
I20260709 13:04:10.351118 11247 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:10.347580 11236 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:10.348592  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:10.313504  9162 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:10.356510  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1758) took 1971 ms. Trace:
I20260709 13:04:10.312049  9192 rpcz_store.cc:276] 0709 13:04:08.977605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.835475 (+857870us) service_pool.cc:224] Handling call
0709 13:04:10.277390 (+441915us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.348898  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1757) took 1965 ms. Trace:
I20260709 13:04:10.349699 11211 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:10.360111  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 46765.6us
I20260709 13:04:10.325961  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:10.307139  9196 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:10.360913 11032 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:10.357028  9191 rpcz_store.cc:276] 0709 13:04:08.384864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.355267 (+1970403us) service_pool.cc:224] Handling call
0709 13:04:10.355728 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.356121 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.356155 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:10.356226 (+    71us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:10.356256 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.356314 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.356435 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.349686 11268 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:10.352427  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:10.352983  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1746) took 1396 ms. Trace:
I20260709 13:04:10.362464  9190 rpcz_store.cc:276] 0709 13:04:08.956647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.343055 (+1386408us) service_pool.cc:224] Handling call
0709 13:04:10.343461 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.346656 (+  3195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.352894 (+  6238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.351528  9266 rpcz_store.cc:276] 0709 13:04:07.907556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.907779 (+1000223us) service_pool.cc:224] Handling call
0709 13:04:10.285147 (+1377368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.286342 (+  1195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.312966 (+ 26624us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000fafd0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:10.313182 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3004406528}
I20260709 13:04:10.353343 11181 raft_consensus.cc:2764] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:04:10.323638  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 677) took 1372 ms. Trace:
I20260709 13:04:10.353549  9194 rpcz_store.cc:276] 0709 13:04:08.554052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.331936 (+1777884us) service_pool.cc:224] Handling call
0709 13:04:10.332319 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.333066 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.333271 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.358358  9197 rpcz_store.cc:276] 0709 13:04:08.383950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.347590 (+1963640us) service_pool.cc:224] Handling call
0709 13:04:10.348093 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.348487 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.348521 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:10.348589 (+    68us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:10.348640 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.348700 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.348841 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.364552  9854 rpcz_store.cc:276] 0709 13:04:08.951542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:08.953643 (+  2101us) service_pool.cc:224] Handling call
0709 13:04:09.723183 (+769540us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:10.323363 (+600180us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54007323d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:10.323546 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273021056}
I20260709 13:04:10.356112 11193 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.360244 11211 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:10.366211 11211 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.366736 11211 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:10.367430  9117 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:10.368046  9162 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:10.369088 11236 consensus_queue.cc:1059] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:10.369892  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.370689 11236 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:10.351785  9198 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:10.361166  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1744) took 1807 ms. Trace:
I20260709 13:04:10.330319  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1762) took 1082 ms. Trace:
W20260709 13:04:10.371526  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.404695  9191 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:04:10.433557  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.445576  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.371590  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.408973  9121 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:10.398783  9117 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:10.485937  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:10.485232  9197 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:10.485136  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1760) took 1412 ms. Trace:
I20260709 13:04:10.428267  9114 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:10.420081  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:10.336083  9018 rpcz_store.cc:276] 0709 13:04:02.130043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.130196 (+   153us) service_pool.cc:224] Handling call
0709 13:04:10.317508 (+8187312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":439750,"reported_tablets":8}
I20260709 13:04:10.487010  9191 rpcz_store.cc:276] 0709 13:04:09.072807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.362339 (+1289532us) service_pool.cc:224] Handling call
0709 13:04:10.404232 (+ 41893us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.484839 (+ 80607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.485080 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.380025  9195 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:10.416042  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.484040  9196 rpcz_store.cc:276] 0709 13:04:08.553780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.284624 (+1730844us) service_pool.cc:224] Handling call
0709 13:04:10.307020 (+ 22396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.360915 (+ 53895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.361089 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.370360 11268 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:10.374023  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:10.371062 11139 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:10.475824  9122 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:10.493309  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.483577  9193 rpcz_store.cc:276] 0709 13:04:09.247978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.205703 (+957725us) service_pool.cc:224] Handling call
0709 13:04:10.330205 (+124502us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.495350  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.439406  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:10.496115  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.496433  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1739) took 2173 ms. Trace:
I20260709 13:04:10.496819  9195 rpcz_store.cc:276] 0709 13:04:08.316541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.272958 (+1956417us) service_pool.cc:224] Handling call
0709 13:04:10.310683 (+ 37725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.489946 (+179263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.490151 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.421569  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:10.372932 10986 raft_consensus.cc:3055] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.499598  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.378010 11256 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:10.371999 11260 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.501822  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:10.504057  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 84153.2us
I20260709 13:04:10.504328  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:10.503549 10986 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:10.371670 11156 raft_consensus.cc:3055] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.399025 11211 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:10.511157  9198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 346) took 1555 ms (client timeout 1650 ms). Trace:
I20260709 13:04:10.371964 11236 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:10.500946 11118 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:10.511722  9198 rpcz_store.cc:269] 0709 13:04:08.953388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:09.617638 (+664250us) service_pool.cc:224] Handling call
0709 13:04:10.502413 (+884775us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:10.517505 11246 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.517748  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:10.518231 11246 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:10.518801  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 147343us
I20260709 13:04:10.383313 11203 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 13:04:10.403442 11032 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:04:10.506502 11260 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:10.508755 11156 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:10.526608  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 81301.2us
I20260709 13:04:10.396669 11231 consensus_queue.cc:1059] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 13:04:10.512760  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.519974 11220 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:10.509147 11139 raft_consensus.cc:697] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:10.512512 11193 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:10.551323 11193 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:10.526255  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:10.551887 11139 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.551915 11193 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:10.553728 11099 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:10.554131 11099 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:10.556631 11231 raft_consensus.cc:3055] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.557138 11231 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:10.557693  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:10.558120 11231 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.520411 11181 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:10.536339 11268 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:10.562783  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:10.563392  9272 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:10.545271  9117 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:10.565634  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 126375us
I20260709 13:04:10.519666  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:10.520179 11210 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.539587 11211 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:10.545188  9828 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.545984 11260 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:10.547055  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 173199us
I20260709 13:04:10.548317 11112 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.548769 11226 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:10.550557  9164 leader_election.cc:336] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Timed out: RequestConsensusVote RPC to 127.7.213.68:43523 timed out after 1.596s (SENT)
I20260709 13:04:10.559515 11231 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.568517  9854 raft_consensus.cc:2376] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2. Re-sending same reply.
I20260709 13:04:10.544390 11220 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.571439  9196 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:10.572708  9197 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 5. Ops: []
I20260709 13:04:10.573638  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.570871  9194 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:10.555116 11099 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.581069  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:10.583720  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 64203.5us
I20260709 13:04:10.334432  9020 rpcz_store.cc:276] 0709 13:04:00.158597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:00.158847 (+   250us) service_pool.cc:224] Handling call
0709 13:04:10.321005 (+10162158us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:05.004958 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:05.005812 (+   854us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:05.005859 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:04:05.005889 (+    30us) tablet.cc:662] Decoding operations
0709 13:04:05.006407 (+   518us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:05.006506 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:05.006550 (+    44us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:04:05.006986 (+   436us) tablet.cc:701] Row locks acquired
0709 13:04:05.007012 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:04:05.007216 (+   204us) write_op.cc:270] Start()
0709 13:04:05.007358 (+   142us) write_op.cc:276] Timestamp: P: 1783602245007145 usec, L: 0
0709 13:04:05.007400 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:04:05.008893 (+  1493us) log.cc:844] Serialized 923 byte log entry
0709 13:04:07.425637 (+2416744us) op_driver.cc:464] REPLICATION: finished
0709 13:04:07.723777 (+298140us) write_op.cc:301] APPLY: starting
0709 13:04:07.728161 (+  4384us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:07.736757 (+  8596us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:07.742424 (+  5667us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:07.781154 (+ 38730us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:07.817339 (+ 36185us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:04:07.831146 (+ 13807us) write_op.cc:312] APPLY: finished
0709 13:04:07.847877 (+ 16731us) log.cc:844] Serialized 29 byte log entry
0709 13:04:07.848064 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:07.848146 (+    82us) write_op.cc:454] Released schema lock
0709 13:04:07.848192 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:07.875249 (+ 27057us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:07.875281 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2111532,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":268157,"apply.run_cpu_time_us":3406,"apply.run_wall_time_us":142011,"num_ops":2,"prepare.queue_time_us":1215791,"prepare.run_cpu_time_us":11653,"prepare.run_wall_time_us":435778,"raft.queue_time_us":553846,"raft.run_cpu_time_us":38504,"raft.run_wall_time_us":146948,"replication_time_us":2418010,"thread_start_us":1898068,"threads_started":4,"wal-append.queue_time_us":468952,"wal-append.run_cpu_time_us":27280,"wal-append.run_wall_time_us":3512719}]]}
I20260709 13:04:10.540900 11118 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.514045 10986 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.513526  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:10.582387 11220 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:10.592579  9195 raft_consensus.cc:606] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:10.556734 11156 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.597167 10931 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:10.598083 10931 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:04:10.600184  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 73844.4us
I20260709 13:04:10.601275  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:10.575371 11193 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.591310 11211 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.601977 11210 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:10.608664 11032 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.617683  9854 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:10.619025 11146 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:04:10.621199  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.615482  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:10.612588  9853 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:10.608863 11112 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.610560 11141 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:04:10.632812  9161 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:10.633598  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.584590 11226 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:10.624155  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:10.622520 11246 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:10.632591 11099 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:10.620625 11268 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:04:10.633980 11260 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:10.644017  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 64095.3us
I20260709 13:04:10.612809 11211 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:10.634933 11210 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:10.599633 10931 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:10.654912  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 124137us
I20260709 13:04:10.641400 11203 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:04:10.656435  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 51274.7us
I20260709 13:04:10.646767 11032 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:10.642642  9266 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
W20260709 13:04:10.674145  9238 leader_election.cc:336] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.655s (SENT)
I20260709 13:04:10.657408 11156 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.679216  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.670257  9854 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:10.685329  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.690088  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.685405 11193 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:10.693827 11193 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.701924 11193 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.645494 11236 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:10.705382 11193 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:10.706758  9856 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:10.694422  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.686791  9853 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:10.722072  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.723176  9857 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 4: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:10.700387 11203 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:10.744861  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 54857.8us
W20260709 13:04:10.747430  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 146336us
W20260709 13:04:10.740964  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 87321.3us
I20260709 13:04:10.738622 11118 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:10.762488 11282 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:04:10.751978 11268 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:04:10.766165 10946 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.766489 11226 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:10.767256  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 343) took 1830 ms. Trace:
I20260709 13:04:10.767657  9189 rpcz_store.cc:276] 0709 13:04:08.936562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.336887 (+1400325us) service_pool.cc:224] Handling call
0709 13:04:10.337366 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.340331 (+  2965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.340373 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.341001 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 13:04:10.498515 (+157514us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:10.730756 (+232241us) log.cc:844] Serialized 14 byte log entry
0709 13:04:10.731044 (+   288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.731143 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.767004 (+ 35861us) raft_consensus.cc:1685] finished
0709 13:04:10.767023 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.767166 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3204,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":7878,"thread_start_us":377381,"threads_started":2,"wal-append.queue_time_us":316803}
I20260709 13:04:10.771224 10986 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.772840 11021 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:10.757715 11181 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:10.778618  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 117271us
I20260709 13:04:10.795671  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:10.774706  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:10.798552  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 49788us
W20260709 13:04:10.800348  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.803351 11247 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:10.804876 11146 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:10.806043 11247 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:10.806545 11247 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:10.806988 11146 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.808456  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.809494  9853 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 2: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:10.817252  9191 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:10.793624 11021 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:10.822674  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.804962  9856 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:10.813733 11211 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:10.828840  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.829457  9857 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:10.801846 11032 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:04:10.837308  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:10.836603  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.827005  9854 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:10.849591  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:10.858238  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 54295.1us
I20260709 13:04:10.813269 11210 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.837670 11236 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:10.874236  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:10.880553  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1753) took 1077 ms. Trace:
I20260709 13:04:10.881152  9191 rpcz_store.cc:276] 0709 13:04:09.803071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.509174 (+706103us) service_pool.cc:224] Handling call
0709 13:04:10.517519 (+  8345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:10.817822 (+300303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.880230 (+ 62408us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54000a6754. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:10.880474 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688797568}
W20260709 13:04:10.884137  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 71695.2us
I20260709 13:04:10.883253 11021 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.887365  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.888262  9856 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:10.890144 11021 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:10.892206  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:10.885139  9238 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.012s (SENT)
I20260709 13:04:10.863046 11260 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:10.834601 11181 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:04:10.903092  9857 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:10.908463 11260 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:10.885926  9853 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:10.911972 11236 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:10.913013  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.916189  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.930233  9862 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:10.918102  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.930702 10890 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:10.931249  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.920840 11220 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.931548  9856 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:10.932302  9861 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:04:10.923509  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 86295.1us
W20260709 13:04:10.924680  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.924523  9161 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.926831  9272 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:10.937139  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.937773  9859 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:10.937584 11247 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.938868  9858 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:10.939874  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.940446  9854 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:10.939904  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 70368.9us
W20260709 13:04:10.943827  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:10.952522  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:10.931460 11246 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:04:10.939653 11247 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:10.948453 11211 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:10.955439  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:10.954216  9089 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:10.957939  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:10.959897  9266 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:10.956135 11156 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:10.950438 11220 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:10.973014  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:10.965185 11283 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.223s	user 0.003s	sys 0.000s
I20260709 13:04:10.966915 11032 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:10.970716  9162 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:10.933202 11226 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:04:10.963485  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:10.974507 11231 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:10.983256 11032 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:10.984735  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1748) took 1232 ms. Trace:
I20260709 13:04:10.975054  9263 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:10.985765 11156 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:10.986472 11156 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:10.985139  9190 rpcz_store.cc:276] 0709 13:04:09.752471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.366250 (+613779us) service_pool.cc:224] Handling call
0709 13:04:10.378938 (+ 12688us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b5800003098. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:04:10.379252 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:10.402875 (+ 23623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:10.402926 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:10.403609 (+   683us) log.cc:844] Serialized 26 byte log entry
0709 13:04:10.636804 (+233195us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:10.636859 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:10.636958 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:10.964440 (+327482us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:10.984501 (+ 20061us) raft_consensus.cc:1685] finished
0709 13:04:10.984524 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:10.984660 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3141,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":127,"spinlock_wait_cycles":32759552,"thread_start_us":232702,"threads_started":1,"wal-append.queue_time_us":276629}
I20260709 13:04:10.987774 11156 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.984505 11032 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.971539 10890 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:10.994153 10890 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:10.994570 10890 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:10.932319 11260 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:10.993407  9162 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:10.998579 11231 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:11.000133  9825 leader_election.cc:336] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.511s (SENT)
W20260709 13:04:11.000877  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 76310us
W20260709 13:04:11.001417  9825 leader_election.cc:336] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.554s (SENT)
I20260709 13:04:11.001314  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:11.002332  9855 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:11.002482  9825 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.568s (SENT)
I20260709 13:04:11.002854  9825 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.002329 11181 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:11.003551 11256 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:11.003996  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 56450.8us
I20260709 13:04:10.970073 11211 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:11.004873  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:11.005532 11211 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:11.006469 11141 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:11.006879 11211 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:11.007193 11141 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:11.007035  9853 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:11.010177  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:11.020035  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:10.963654  9854 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
W20260709 13:04:11.029610  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 113834us
I20260709 13:04:11.010102  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:11.027163 11226 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:11.038059  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.030215 11220 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:11.041034  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.043196  9116 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:11.043264  9825 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.573s (SENT)
I20260709 13:04:11.043915  9825 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.036899 11210 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:11.046137  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.038861 11226 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.046130  9855 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:11.051621  9117 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:11.051918 11292 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
W20260709 13:04:11.062543  9069 auto_rebalancer.cc:893] Removing replace marker failed: Replica 94504d29c6744f38bb06943479df9466 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:04:11.063083  9069 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:04:11.063427  9069 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:04:11.051126 11226 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:11.045915 11210 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:04:11.074112  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43559us
I20260709 13:04:11.080801 11303 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:11.083240  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.084900 11102 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:11.087787  9263 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:11.087877 11146 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:11.096444  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.092216  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:11.098011  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 51963.9us
I20260709 13:04:11.098559  9122 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:11.096848 11210 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:11.107551  9118 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:04:11.109280  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.096581 11102 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:11.122941 11292 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:11.125188 11292 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:11.124850 11102 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:11.125617 11292 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:11.126048 11102 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:11.112586  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 80561.7us
W20260709 13:04:11.114039  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.117017  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.120755  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:11.122884 11211 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:11.123418 11099 raft_consensus.cc:3055] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:11.123085 11112 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:11.096481 11139 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:11.135017 10946 consensus_queue.cc:1059] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:11.126672 11292 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:11.141741  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 96554.5us
W20260709 13:04:11.142198  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 45831.4us
I20260709 13:04:11.142586  9116 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:11.145488 11256 raft_consensus.cc:3055] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:11.145921 11308 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:04:11.147300 11308 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:11.147692  9114 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:11.148109  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.148473  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.148622 11292 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.147693 11308 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:04:11.147863  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.149952 11308 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:11.150847  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.151444 11308 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:11.152307 10890 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.153263  9161 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.148823  9122 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:11.137627 11099 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:11.141160 11141 consensus_queue.cc:1059] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
W20260709 13:04:11.169320  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48739.4us
I20260709 13:04:11.171931  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.172652  9116 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:11.153285 11236 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:11.171085  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.175397  9123 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:11.178310  9116 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:11.165179 11211 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:11.185920  9122 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:11.145980 11256 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:11.188871  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.196666  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.203436  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.204082  9120 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:11.204348  9825 leader_election.cc:336] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 2.018s (SENT)
I20260709 13:04:11.205693 11301 raft_consensus.cc:993] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:04:11.209103  9119 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:11.195182 11203 raft_consensus.cc:3055] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:11.168327 11303 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:04:11.212941  9122 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:11.218940  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.209542 11211 raft_consensus.cc:1081] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
W20260709 13:04:11.232371  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 67289.6us
I20260709 13:04:11.236147  9117 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
W20260709 13:04:11.236388  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 77082us
I20260709 13:04:11.212005 11203 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:11.217667  9020 catalog_manager.cc:5697] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.232007 11032 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.264341  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.292376  9118 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:11.286199  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:11.272081 11313 raft_consensus.cc:3055] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:11.296092  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:11.296190 11313 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:11.297562  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:11.297760 11313 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:11.274986  9121 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:11.261407 11246 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:11.305673  9189 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:11.307554  9161 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.308025  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:11.300113  9020 catalog_manager.cc:5697] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:11.299714 11313 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:11.237555 11236 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
W20260709 13:04:11.315487  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.332398  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 91763.6us
I20260709 13:04:11.338092  9191 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
W20260709 13:04:11.340518  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 121643us
I20260709 13:04:11.341920  9164 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:11.346467  9272 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:11.348371  9191 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:11.351770  9191 raft_consensus.cc:606] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:11.314108 10890 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:11.314435 11211 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:04:11.362975  9020 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.358764 11319 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:11.368688  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:04:11.356170 11303 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:04:11.371902  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:11.372649  9266 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:11.374459  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.373679  9115 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:11.371549  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:11.362021 11112 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:11.393548  9020 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 3 to 5, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:11.400630  9020 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:11.402060  9020 catalog_manager.cc:5697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 4, VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) evicted. New cstate: current_term: 2 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:11.405055 11315 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:11.405718 11315 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.407801 11317 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:11.407635 11099 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.408944 11315 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:11.411841  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.412899  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.413532  9114 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:11.414103  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.414824  9120 raft_consensus.cc:2376] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2. Re-sending same reply.
I20260709 13:04:11.415232  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.424928 11303 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:11.425436 11303 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:11.425949  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49438.9us
I20260709 13:04:11.426371 11303 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.334323 11203 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:11.434458  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.393663 11112 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:11.396289 11319 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:11.437914  9116 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:11.427886 11318 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:04:11.438949  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.375816 11210 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.394346 10946 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:04:11.437609 10890 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:11.447408  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.448094 11203 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.453528 11210 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:11.442701 11112 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:11.427700 11303 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:11.455892 11328 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:04:11.471469  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 107060us
I20260709 13:04:11.472596 11301 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:11.473121 11301 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:11.472412 11325 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:11.473922  9020 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 3, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:11.477267 11156 raft_consensus.cc:993] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:04:11.423457 11226 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:11.488734  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:11.489410 11325 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.494041 10986 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.495893  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:11.496464 11156 raft_consensus.cc:1081] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:04:11.496063 10986 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.497915  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.500994  9123 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:11.506364  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 78085us
W20260709 13:04:11.506467  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 242237us
I20260709 13:04:11.507505  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.503291  9020 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 5 to 7, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 7 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:11.495798 11313 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:11.498647 10986 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:11.520140  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.504562 11320 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:11.521268  9116 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:11.485266 11318 consensus_queue.cc:1243] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:11.515848  9020 catalog_manager.cc:5697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 4 to 5, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 5 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:04:11.524513  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 96346.9us
I20260709 13:04:11.525586  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.533912  9020 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:04:11.537048  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.472604 11256 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:11.520639 11141 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:11.493124 11319 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:04:11.537632  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.552381  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.552868  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.553145  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.540452  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:11.526363 11099 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:11.556694 11330 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:11.557384 11330 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:11.541312  9120 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:11.515522 11210 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:11.542836  9115 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:11.545186 11226 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:11.546836 11334 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:11.547428  9164 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:11.547817  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.549816 11326 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.520920  9263 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:04:11.499431 11328 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.563998 11325 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:11.581606  9114 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:11.559288 11330 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
W20260709 13:04:11.569060  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 54683.7us
I20260709 13:04:11.537833 11313 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:11.583163  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:11.585973  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.565620 11334 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:11.590639  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:11.597187  9264 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:11.590857  9116 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:11.591717 11313 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.592739  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.594875  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.572026 11156 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.587584  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:11.604359  9115 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:11.585358  9263 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:11.565387 11210 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:11.610042  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 70935.6us
I20260709 13:04:11.596352 11211 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:11.523628 11301 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.612908  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:11.615480  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:11.616132  9270 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:11.618167  9271 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:11.615582  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.619020  9116 raft_consensus.cc:2376] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5. Re-sending same reply.
I20260709 13:04:11.612658  9263 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:11.626286 11320 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:11.626784  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:11.629658  9269 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:11.587908 11326 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.632404  9118 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:11.638370 10890 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:11.640317 10890 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:11.640756 10890 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:11.609014 11156 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:11.649155  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.650774  9120 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:11.651034 11156 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:11.653359 11326 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:11.654376 11246 raft_consensus.cc:993] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:04:11.656920  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.645509  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.648077  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.657662  9114 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:11.648864  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.658588  9122 raft_consensus.cc:2376] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2. Re-sending same reply.
I20260709 13:04:11.666898  9265 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:11.669071  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.669649  9118 raft_consensus.cc:2376] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2. Re-sending same reply.
I20260709 13:04:11.673580  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:11.674134  9089 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:11.674396  9116 raft_consensus.cc:2376] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2. Re-sending same reply.
I20260709 13:04:11.678759  9267 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:11.679219  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:11.675504 11303 raft_consensus.cc:1081] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:04:11.685277  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.686115  9116 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:11.689795  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:11.691932  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:11.692381 11328 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:11.692749 11317 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:11.694164 11317 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:11.692481  9191 raft_consensus.cc:2376] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2. Re-sending same reply.
I20260709 13:04:11.678822 11102 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:11.695969  9267 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:04:11.696909 11337 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:11.656199 11334 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.717952  9270 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
W20260709 13:04:11.733871  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 86120.9us
I20260709 13:04:11.797027  9271 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:11.833098  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:11.672425  9266 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:11.733604  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:11.844164  9191 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:11.774871  9123 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:04:11.845106  9123 pending_rounds.cc:85] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 1
I20260709 13:04:11.642190 11323 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:04:11.701812 10946 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:11.723583  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:11.641744 10890 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.846292  9089 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:11.768432  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:11.784080  9859 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:11.780579  9196 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:11.743332  9267 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:04:11.759841  9192 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:04:11.769071  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.796627  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.792783  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:11.735364  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:11.734553  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:11.754585  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:11.804395 11112 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:11.806286 11347 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:04:11.800973  9853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:04:11.808924 11210 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:11.815304  9264 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:11.740329 11339 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:04:11.755326  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:11.819547  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.835806  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:11.840112 11220 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:11.841235  9272 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:11.845458  9123 pending_rounds.cc:107] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:04:11.862478  9123 raft_consensus.cc:2889] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:11.736261 11246 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:04:11.868273  9268 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:04:11.845425  9271 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:11.853264 11256 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:11.855517  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 87200.6us
I20260709 13:04:11.798132 11320 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.832516 11099 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:11.844271 11317 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:11.850313 10946 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:11.851738  9267 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:11.878738  9267 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.857839  9855 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:11.857012 11032 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:11.874351  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:11.869449  9853 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:11.870983 10931 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:11.883388  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:11.884167 10946 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:11.899039  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:11.902874  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:11.889807  9853 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:11.908936 11032 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:11.897012 10890 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:11.912673 11348 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:04:11.877476 11220 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.917873  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:11.883157 11099 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:11.887189  9263 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:11.938884  9190 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
W20260709 13:04:12.122579  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 239243us
W20260709 13:04:11.959208  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 83815.3us
W20260709 13:04:12.140995  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.173391  9828 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:12.174793  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:12.064498  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:11.961247  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.094501  9267 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:12.180327  9271 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:12.179432  9189 raft_consensus.cc:2410] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:12.175851  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:11.960122 11346 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 13:04:11.994347 11350 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:04:12.197984  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:12.216517  9267 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:12.217762  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:12.122352 11226 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.122112 11324 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:12.122597 11317 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:11.946291 10931 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:12.230119  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:12.233129  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:12.199537 11318 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.195063 11313 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.202649 11112 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:12.251668  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 78379us
I20260709 13:04:12.249749  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:12.229768  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:12.252851  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:12.254218  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.297780  9269 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:12.297946  9267 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:12.234653 11226 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.235589 11324 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.243465 11317 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.330403  9269 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:12.252543 11318 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:12.251441 11313 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:12.352445  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.335210 11329 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:12.359398  9269 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:12.331313 11357 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.363409  9272 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:12.367846  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 115152us
I20260709 13:04:12.379303  9268 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:12.382231  9269 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:12.383648  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.503486  9854 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:12.381107 11357 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.556521  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:12.557575 11357 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:12.558156 11357 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.558719  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:12.558656 11357 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.530709  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:12.362299 11220 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:12.529497  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:12.570302  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 305791us
I20260709 13:04:12.508692 11341 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:12.402109  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.490753  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:12.530251  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 282181us
I20260709 13:04:12.564610 11357 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:12.586932 11220 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.366935 11356 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:12.589071 11220 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:12.507632 11362 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:04:12.360622 11360 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.449535 11099 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:12.491219  9263 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:12.507010 11366 log.cc:927] Time spent T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:04:12.563300 11361 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.385762  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:12.560289  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:12.570453  9860 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:12.589318 11324 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:12.591403 11220 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:12.362628 10931 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.604557 11328 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:12.605142  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.562489  9857 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:12.605967 11364 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 13:04:12.589334 11358 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:04:12.601740  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:12.605608  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.617005  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.617354  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.605077 11328 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:12.617647  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.618276 11363 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 13:04:12.619168 11328 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:12.611547 11032 consensus_meta.cc:287] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.702s	user 0.007s	sys 0.000s
I20260709 13:04:12.624106 11032 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:12.626803 10946 consensus_meta.cc:287] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.718s	user 0.007s	sys 0.000s
I20260709 13:04:12.628007 11032 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.628805 11301 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:12.617040  9853 consensus_meta.cc:287] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.000s
I20260709 13:04:12.632278  9853 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.633837  9859 raft_consensus.cc:2393] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 2: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:12.587234 11236 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.628537 11032 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.594667 11210 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:12.587369  9854 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:12.612725  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:12.596588 11323 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:12.652545  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:12.648072  9191 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:12.648646  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:12.614359 11315 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.627286 10946 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.599365 11361 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:12.601012  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:39752 (request call id 1878) took 1099 ms. Trace:
I20260709 13:04:12.503520 11367 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.256s
I20260709 13:04:12.671790  9193 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:12.672217  9190 raft_consensus.cc:3055] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:12.684126  9190 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:12.639850 11256 consensus_meta.cc:287] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.756s	user 0.007s	sys 0.004s
I20260709 13:04:12.687947 11032 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:12.690968 10946 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:12.691499  9267 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 4. Ops: []
I20260709 13:04:12.693001 11318 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:12.693428  9196 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:12.677419 11323 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:12.595652 11341 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:12.642457 11334 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:12.685021  9190 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.648797  9853 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:12.700714  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 101252us
W20260709 13:04:12.703683  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.649147 11370 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:12.704524  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 85747.9us
I20260709 13:04:12.670423 11226 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:12.649847 11236 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:12.715438  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 111246us
I20260709 13:04:12.638533 11356 consensus_queue.cc:1059] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:04:12.717562  9862 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:12.719848  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:12.726409  9114 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:12.597980 11360 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.720194  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:12.674628 11210 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.723238  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1885) took 1016 ms. Trace:
I20260709 13:04:12.685576 11361 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.732750  9270 rpcz_store.cc:276] 0709 13:04:11.707249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.833320 (+126071us) service_pool.cc:224] Handling call
0709 13:04:11.833633 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:11.847129 (+ 13496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:11.847175 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:11.850835 (+  3660us) log.cc:844] Serialized 26 byte log entry
0709 13:04:12.214818 (+363983us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:12.214923 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:12.215066 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:12.722982 (+507916us) raft_consensus.cc:1685] finished
0709 13:04:12.723008 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:12.723177 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5292,"raft.queue_time_us":15884,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":7804,"thread_start_us":351205,"threads_started":1,"wal-append.queue_time_us":469222}
I20260709 13:04:12.739109 11318 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:12.707129 11032 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:12.740175 11032 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:12.742084 11361 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.742570 11032 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:12.745169  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.643867 11371 consensus_queue.cc:1059] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.126s
I20260709 13:04:12.714251 11317 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:12.745443  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:12.723909  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:12.683930  9263 rpcz_store.cc:276] 0709 13:04:11.502432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.505043 (+  2611us) service_pool.cc:224] Handling call
0709 13:04:12.599126 (+1094083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:12.680155 11315 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.726794 11256 raft_consensus.cc:1064] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:04:12.671041 11365 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 13:04:12.760353  9162 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:12.761989  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.749261  9855 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.850s	user 0.012s	sys 0.000s
I20260709 13:04:12.753859 11313 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:12.724961 11370 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.746047 11318 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:04:13.271155 11301 consensus_meta.cc:287] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.642s	user 0.004s	sys 0.008s
W20260709 13:04:13.138672  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.269512 11361 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:12.880689  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 160580us
I20260709 13:04:12.777616 11319 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:13.280208  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 519938us
W20260709 13:04:13.269845  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 546085us
I20260709 13:04:13.281333  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.281544  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:13.273604 11384 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.273518  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:13.282688 11319 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:13.282848 11384 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:12.822268  9114 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
W20260709 13:04:13.004168  9826 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.268s (SENT) [suppressed 67 similar messages]
I20260709 13:04:12.770493 10931 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.274519 11370 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.308306 10931 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.308905 11384 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.310381 10931 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:13.269865 11380 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.310657 11370 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.311131 11380 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.311308 11370 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.313453 11380 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:13.270062  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 621476us
I20260709 13:04:13.313812 11370 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:13.314608 11365 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:13.315416 11334 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.003s
I20260709 13:04:13.315531  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:13.318372 11379 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:13.318815 11379 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:13.319909 10986 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.320168  9195 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:13.322022  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1780) took 1152 ms. Trace:
I20260709 13:04:13.322365  9195 rpcz_store.cc:276] 0709 13:04:12.169624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.169810 (+   186us) service_pool.cc:224] Handling call
0709 13:04:12.197648 (+ 27838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:13.320512 (+1122864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.321751 (+  1239us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54004178d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:13.321952 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2580874112}
I20260709 13:04:12.817056 11378 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.323856 11378 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.326025 11378 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:12.772176 11317 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:12.783561 11315 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:12.781831 11323 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.319543  9089 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:13.335001 10890 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:12.919453  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:12.784384 11377 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:04:12.893482  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:12.873420 11341 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:12.783951  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:13.354671  9193 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.358879 10890 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.361157 11384 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:13.361811 11384 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:13.175966 11356 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:12.816696  9856 consensus_meta.cc:287] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.001s
I20260709 13:04:12.780717 10946 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.271551  9855 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:13.363306 11356 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.363528 10946 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.363351  9856 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:13.364804  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 665) took 1162 ms. Trace:
I20260709 13:04:13.365163  9856 rpcz_store.cc:276] 0709 13:04:12.201986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.212215 (+ 10229us) service_pool.cc:224] Handling call
0709 13:04:13.364741 (+1152526us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:13.365840 11356 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.366044 10946 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:12.779058 11381 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:04:12.785825 11236 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.367986  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:13.368800  9268 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:13.369313  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:12.761828 11102 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:13.370762  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.371337  9264 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:13.284014 11337 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:13.372051 11337 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:13.372514  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.373122  9264 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:13.373039 11337 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.284005  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.375425  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:12.753525 11357 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:13.376224 10946 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:13.376340  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 92422us
I20260709 13:04:12.781934 11210 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:12.813104 11226 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:12.779301 11374 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:13.377396  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.377899 11374 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.378595  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.379360 10946 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:12.761788 11032 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:13.138851  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.290009 11388 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.053890 11360 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:13.037241  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.376657 11357 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.380120  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.367995  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.037851 11367 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.384006 11367 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.379698  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:13.385480 11320 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.380456 11032 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.388980 11367 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:12.778928 11333 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.397526 11333 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.292382 11390 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:12.873284 11211 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:13.400199 11211 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.400928  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:13.401438  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.402433  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.402587 11211 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.296633  9087 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:13.301100  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 476) took 1409 ms. Trace:
I20260709 13:04:12.995409 11303 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:13.403764  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 107224us
I20260709 13:04:13.391096  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.404560  9087 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:13.404595  9264 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:13.404857  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:13.403335  9267 rpcz_store.cc:276] 0709 13:04:11.891679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.303146 (+411467us) service_pool.cc:224] Handling call
0709 13:04:12.331482 (+ 28336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:12.702283 (+370801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.300813 (+598530us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54000fa5d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:13.301029 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":827559680}
I20260709 13:04:13.405689  9856 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
W20260709 13:04:13.406212  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 402082us
I20260709 13:04:13.407052 11303 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:13.408970  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.409402 11356 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:13.409891 11356 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:13.410419 11303 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.410631 11102 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:13.410802 11356 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.331276 11301 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.412286 11301 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.412775  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.412997  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.413599  9197 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:13.414713 11301 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.329057 11323 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:13.416114 11323 raft_consensus.cc:3055] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:13.416616 11323 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:13.418262  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:13.330329 11317 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.344775  9857 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:13.345894  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:13.420382  9860 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:13.346674 11370 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.421198  9856 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:13.421314 11370 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.419220  9862 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 4: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:13.347886 10931 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.411729 11400 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:13.424180 10931 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.426573 10931 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.427081 10931 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.429116 11402 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:13.348747 11359 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.434944 11378 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.434850 11359 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.435316 11400 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:13.435500 11378 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.436424 11400 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.437626 11378 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:13.349447 11379 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.333271  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1779) took 1177 ms. Trace:
I20260709 13:04:13.362366  9193 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:13.439085 11378 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:12.880182  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:13.439692 11378 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.441740 11378 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:13.362713  9270 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:13.439493 11402 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.444164 11402 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:13.283432 11319 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:13.445091  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.445326 11319 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.284116 11313 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:13.447494  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.448263 11313 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:13.444679  9191 rpcz_store.cc:276] 0709 13:04:12.156118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.156318 (+   200us) service_pool.cc:224] Handling call
0709 13:04:12.647826 (+491508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:12.672440 (+ 24614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.333019 (+660579us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400218250. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:13.333195 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1042553728}
I20260709 13:04:13.389518 11341 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:13.449596 11341 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.449797  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.450048 11341 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:13.450620  9263 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:12.874892 11203 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:13.452525  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:13.452256 11203 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.440753 11406 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:04:13.389962 11032 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.454761  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.455327 11203 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.390069 11210 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:13.381157 11360 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:13.455984 11400 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.456188 11360 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.456800 11400 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.458472 11360 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:13.381702  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.459007 11333 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.379840 10946 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:13.380191 11374 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.380846 11388 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.459896 11400 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.383514 11357 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.461140  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:13.381032  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.461884  9197 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:13.462347  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.459987 11374 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:13.462934  9116 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:13.463315 11374 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.463841 11374 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:13.464057  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:13.464363  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.464633  9114 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:13.466928  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.411212 11102 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.468513  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.427573 10931 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.469182  9121 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:13.469491 11102 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.417760 11323 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.470602 11323 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.439145 11379 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:13.397133 11384 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.461891 11388 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.430239  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:13.431577 11370 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:12.878186 11256 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.476333 11210 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.476996  9114 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:13.476925 11210 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.433817  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:13.434168 11315 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.438921 11359 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.443382  9235 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.363922  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 660) took 1591 ms. Trace:
I20260709 13:04:13.448694 11313 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:13.290084 11318 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.377390 11226 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:13.459712 10946 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.459039 11360 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:13.480218  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.494004 10890 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.494650  9121 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 2: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:13.480361 11315 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.480628  9855 rpcz_store.cc:276] 0709 13:04:11.772086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.774662 (+  2576us) service_pool.cc:224] Handling call
0709 13:04:13.363853 (+1589191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:13.497665  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.498056 11315 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:13.481549  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.498626  9266 raft_consensus.cc:2410] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:13.498684 11388 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.499125  9116 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 1.
I20260709 13:04:13.499384 11388 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.500075  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.467483  9191 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:13.482318 11226 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:13.501739 11388 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.479846  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.503602  9263 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:13.482535 11378 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:13.485008  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.504758 11378 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:13.505767  9235 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.419210 11317 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.507181  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.508123 11382 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:13.508535 11382 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.508872 11382 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:13.509366 11317 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.506063 11378 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.512068 11387 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.512522 11387 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.512850 11387 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.348331 11361 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.489632  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.470561 10931 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:13.514843 11361 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.487524  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.516242  9115 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:13.484097  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.517087  9853 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:13.489295  9119 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:13.518066  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.480111  9862 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:04:13.481343 11313 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.491814 10946 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.520679 11313 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:13.521581 10946 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.524420 11328 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:13.524947 11328 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:13.525795 11328 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.483340 11360 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:13.529048 11361 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.483637  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.485608 11210 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:13.528868 11360 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.536955  9235 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.539129 11402 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.541307 11210 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.541801 11210 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.542258 11210 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.514334  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.550580  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.553006 11320 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.554001  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.555845 11328 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.556308 11328 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.556056 11378 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:13.556841 11328 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.564324  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:13.542251 11402 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.565174  9860 raft_consensus.cc:2410] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:13.569053  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:13.569455  9825 leader_election.cc:336] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.580s (SENT)
I20260709 13:04:13.569689  9117 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:13.569914  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.572909  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.573474  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.574360  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.564873  9861 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:13.575802  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 698) took 1190 ms. Trace:
I20260709 13:04:13.576122  9861 rpcz_store.cc:276] 0709 13:04:12.385081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.385278 (+   197us) service_pool.cc:224] Handling call
0709 13:04:13.564414 (+1179136us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:04:13.564579 (+   165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:13.575511 (+ 10932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:13.575704 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48010624}
I20260709 13:04:13.576893  9162 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.577361  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.577677 11317 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.577932  9116 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:04:13.578183 11317 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.573118  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.579981  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.580330 11317 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.580857 11317 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.581190 11317 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.581513 11317 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.581640  9123 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:13.586131  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.588207  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.588879  9267 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 1.
I20260709 13:04:13.590188 11339 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.590879 11339 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.592082 11387 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.593357  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.594038 11339 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.594911  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.595322  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.595515  9119 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:13.595587  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.596345  9123 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:13.596576  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.596773  9164 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:13.598157 11318 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:13.598676 11318 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:13.598515 11333 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:13.599489 11323 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.599591 11318 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.600360 11323 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:13.603371  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.603685 11323 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.605080 11203 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:13.605955  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:13.605674 11203 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:13.608480  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:13.609050  9193 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:13.609380  9193 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:13.610342 11203 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:13.610071  9193 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.611289  9193 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:13.612764  9197 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:04:13.613711 11382 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.616341  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.616657  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.617251  9195 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:13.618716  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.619292  9195 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:13.620123  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:13.621583  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.623262  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.623416  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.623909  9196 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:13.625180  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.625744  9196 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:13.628180  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.628886  9196 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:12.992019 11371 raft_consensus.cc:3055] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:13.019349 11376 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.631275  9162 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.640647  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:13.641676  9193 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:13.410930  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:13.603380  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.654620  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.656060  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:13.656672  9119 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:13.655537  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:13.659138  9117 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.663239  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.664002  9114 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:13.462707 11406 raft_consensus.cc:3055] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:13.665181  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:13.666091  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 62782.5us
I20260709 13:04:13.667956  9196 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:04:13.668300  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.661722 11376 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.670821 10986 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.671290 11315 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.671444  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.695304  9267 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:13.672773 11361 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:13.673629  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 104207us
W20260709 13:04:13.674973  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 264103us
I20260709 13:04:13.675499 11360 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:13.679898  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.684403  9190 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.686120  9269 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:13.686869  9164 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.689227  9119 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:13.691690  9191 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:13.637056  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.720214  9117 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:13.636837 11371 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:13.705209  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.661530 11334 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:13.703485 11226 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:04:13.713510  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.714214 11323 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:13.732703  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 69835.9us
I20260709 13:04:13.665978 10946 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:13.691787 11387 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:13.725715  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 323353us
I20260709 13:04:13.691126 11203 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:13.730198  9190 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:13.686789 11406 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:13.679436 11328 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:13.757642  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:13.732730 11419 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:13.744561 11203 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:13.753634  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 368) took 1091 ms. Trace:
W20260709 13:04:13.719470  9825 leader_election.cc:336] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.634s (SENT)
I20260709 13:04:13.758504  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:13.758410  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 90189.3us
I20260709 13:04:13.758320  9190 rpcz_store.cc:276] 0709 13:04:12.649702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:12.652196 (+  2494us) service_pool.cc:224] Handling call
0709 13:04:13.661123 (+1008927us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b58001568d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566400aa 00007f76566c2447 00007f76566dafe7 00007f76566dc7de 00007f76566e827c 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c
0709 13:04:13.740641 (+ 79518us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389259008}
I20260709 13:04:13.731397 11334 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.760675  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.736320 11323 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.736613 11301 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:13.764977  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.767860  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:13.762115  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:13.771013  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 121172us
I20260709 13:04:13.772639  9270 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:13.779124  9162 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.773402  9235 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.707881 11315 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.767068  9263 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:13.769675 11356 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.755120 11382 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.759574 11419 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.762565 11203 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:13.780340  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.764672 11319 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:13.782166  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 76565us
I20260709 13:04:13.768309 11334 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:13.772814 11323 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:13.769905 11380 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:13.804168 11380 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.787581 11406 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:13.805805 11382 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:13.806164 11406 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:13.806362 11382 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:13.806449 11380 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.781466 10986 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:13.807403 11382 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.792467 11256 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:04:13.781745  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:13.788393  9828 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.779232 11387 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:13.796203  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:13.796820  9825 leader_election.cc:336] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.650s (SENT)
I20260709 13:04:13.794219 11419 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:13.817312  9825 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.817s (SENT)
I20260709 13:04:13.818553  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.793656  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:13.824757 11323 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.828106 11323 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:13.760286 11399 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:04:13.788542 11292 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:13.895478  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.896409  9119 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:13.897320  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.781690 11370 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:13.869138  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:13.842051  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48491.1us
I20260709 13:04:13.899859  9197 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:13.829880  9162 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:13.888468  9196 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:13.862146  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.900919  9193 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:13.836340  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.861862  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.901674  9114 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:13.786880 11333 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:13.779760 10931 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:04:13.901847  9191 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:13.811811 11226 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:13.909170  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:13.923614  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.924459  9117 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:13.830173 11380 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:04:13.885305 11401 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:04:13.777486 11317 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.820824 11360 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:13.920133  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 90341.6us
I20260709 13:04:13.937282  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.938050  9162 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:13.938479  9269 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:13.904338 11292 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.885372 10890 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:13.909373  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:13.925565  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:13.925578  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 152239us
I20260709 13:04:13.929956  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.931500  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.820869 11318 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:13.963788  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 175462us
I20260709 13:04:13.929868 11323 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:13.788062 11359 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 13:04:13.921399  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.922755  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:13.895334 11387 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:13.951149  9268 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:13.953804  9269 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:04:13.910205 11210 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.959084  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:13.782544 11371 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.932209  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:13.933761 11032 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:13.994278 11403 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:04:13.927136 11378 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:13.996021 11378 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:13.998155 11378 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:14.000056 11328 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:14.001104 11408 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:04:13.955732 10890 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:13.812343 11356 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:13.984678  9195 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:13.959060  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:14.010744  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 51735.4us
I20260709 13:04:14.011686  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.013435  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.950119 11360 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.988538  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.965010  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:13.990077  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.014894  9119 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:13.957123 11292 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.991953  9117 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:13.978628 11319 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:04:13.977586  9272 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:14.006561  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.963790 11318 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:13.964097 11333 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:14.026226 11318 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:13.973213  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.020243  9120 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:14.027402  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.028228  9120 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:14.029007  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.030049  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.031085  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:14.020464  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.022670 11371 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.024387 11317 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:14.025514  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:13.964193 11402 raft_consensus.cc:2764] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:14.031909  9119 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:13.991771 11376 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:14.035333  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 70382.9us
I20260709 13:04:14.045326  9120 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:13.972044 11380 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:04:14.009140 11359 consensus_queue.cc:1243] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:13.960116 11370 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:04:14.016209 10890 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:14.072101  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.074100  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:14.074420  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 136495us
I20260709 13:04:14.020044 11360 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:14.077687 11417 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
I20260709 13:04:14.080245  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.031930 11320 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:14.035432 10946 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:14.036170 11378 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:14.035063 11395 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.095850  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.027415 11203 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:14.132275  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 118941us
I20260709 13:04:14.132441  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.102353  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.076210  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:14.136732  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:14.115617 11313 consensus_meta.cc:287] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.003s
I20260709 13:04:14.113371 11359 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:14.146373  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 66232.2us
I20260709 13:04:14.115052 10946 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:14.118140  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 297389us
I20260709 13:04:14.122468  9162 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.147717  9089 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:14.124598 11430 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:14.126008  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.126816  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.128747  9860 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:14.119650 11378 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:14.130360  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.132113  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:14.131670  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 314409us
W20260709 13:04:14.115890 11374 consensus_meta.cc:287] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.652s	user 0.005s	sys 0.005s
I20260709 13:04:14.143491 11315 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.115844 11388 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:14.131843 11424 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:14.130199  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.161507  9862 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:14.131275  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.139261 11395 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:14.164857  9825 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.921s (SENT)
I20260709 13:04:14.191973  9825 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.192209 11395 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:14.142603 11320 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:14.110100  9272 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:14.156456 10946 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:14.174832 11388 consensus_queue.cc:1059] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:14.175390 11203 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.152218 11387 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.175818 11422 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:14.176050 11391 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:14.236124 11391 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:14.236563 11391 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.176544 11383 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.240360 11356 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.240274 11383 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.242586 11383 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.177050 11423 consensus_queue.cc:1059] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.132s
I20260709 13:04:14.193400 11395 consensus_queue.cc:1059] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:04:14.244338 11423 raft_consensus.cc:3055] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:14.244706 11423 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:14.245672 11423 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.247008 11423 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:14.178658 11318 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.249375 11318 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.168444  9854 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:14.168628 11333 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.171523  9856 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:14.180317 11429 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 13:04:14.208102 11379 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.737s	user 0.007s	sys 0.003s
I20260709 13:04:14.252440  9118 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:04:14.252781  9118 pending_rounds.cc:85] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 5
I20260709 13:04:14.252990  9118 pending_rounds.cc:107] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.6
I20260709 13:04:14.253244  9118 raft_consensus.cc:2917] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Aborting config change with OpId 4.6: Aborted: Op aborted by new leader
I20260709 13:04:14.253574  9118 raft_consensus.cc:2889] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.253875  9118 pending_rounds.cc:107] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:04:14.254092  9118 raft_consensus.cc:2889] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.207330 10931 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:14.256085 11301 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:14.164207 11378 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.257118 11437 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.257786 11437 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.183028 11424 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:14.259984 11424 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.184581 11032 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:14.264350 11032 raft_consensus.cc:3055] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:14.264932 11379 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:14.164497 11370 consensus_queue.cc:1243] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:14.265586 11379 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.264822 11032 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:14.197868  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:14.156661 11430 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:14.222905 11292 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:14.209766  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 133635us
W20260709 13:04:14.274721  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:14.210237  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49844.7us
I20260709 13:04:14.276073  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.276566  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.277289  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.277983  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.279110  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.278905 11360 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.224200 11426 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.279817  9197 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:14.212826  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 90439.9us
I20260709 13:04:14.225670  9263 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:14.226831 11320 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.235532 11422 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:14.244781 11395 raft_consensus.cc:3055] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:14.180239 11210 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:14.172924 11374 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.179512 11425 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.256313 10931 raft_consensus.cc:697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:14.186138 11419 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:14.300221 11395 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:04:14.270496  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72692.1us
W20260709 13:04:14.281821 11341 consensus_meta.cc:287] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.831s	user 0.011s	sys 0.000s
I20260709 13:04:14.276818  9189 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 2: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:14.277390  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.321707  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:14.322274  9191 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:14.216928 11323 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:14.280848  9162 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:14.234278  9120 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:14.239524 11334 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:14.306901 11382 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:04:14.263270 11391 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:14.308889  9198 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:14.308507 11032 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.310243 11426 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.317761  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:14.207394  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 59785.6us
W20260709 13:04:14.320662  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44153.5us
I20260709 13:04:14.273066 11356 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:14.301364  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 108119us
I20260709 13:04:14.315874  9856 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:14.345928  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.344727  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:14.317447  9854 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:14.323668 11374 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:14.336639  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.287549 11360 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:14.337690  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56916.1us
I20260709 13:04:14.353201 11379 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:14.338558  9825 leader_election.cc:336] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.991s (SENT)
I20260709 13:04:14.355825  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.313504 11395 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.337822 11425 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:14.339790 11430 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.340929 11391 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:14.341697 11032 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.344151  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.344090  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.344699 11371 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:14.344938  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:14.372359  9861 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:14.345543  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.373656  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:14.373864  9857 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:14.313596 11419 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:14.338394  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:14.328804 10986 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:14.350103  9860 raft_consensus.cc:2368] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7.
I20260709 13:04:14.312803 10931 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.343452 10946 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:14.370541  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.371688  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:14.372159  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:14.348289  9854 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:14.329123 11210 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:04:14.335110  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.376029  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:14.376561  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:14.370610  9825 leader_election.cc:336] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 2.044s (SENT)
I20260709 13:04:14.375072  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.379884  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:14.428907  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 52584.8us
I20260709 13:04:14.358918 11425 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.382146  9272 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:14.383157  9271 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:14.368338 11360 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.347210 10890 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:14.338137 11323 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.386538  9268 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:14.388886 11421 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.389364 11420 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:14.393534  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.383502  9855 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:14.392273 10986 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:14.408663  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.379016 11356 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:04:14.412726  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:14.441423  9860 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:14.441834  9122 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:14.406411 11437 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.392200 11395 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:14.381038 11429 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:14.427958  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 57377.9us
I20260709 13:04:14.329502 11341 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.284123 11378 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:14.429970  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:14.431872  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:14.485829  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55937.3us
I20260709 13:04:14.433459  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:14.493558  9857 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.379096  9266 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:14.494095  9264 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:14.384894 11430 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:14.438661  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:14.381346 11303 consensus_queue.cc:1059] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:04:14.443342  9271 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:04:14.445624  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 110635us
I20260709 13:04:14.434383 11339 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.433247 11425 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:14.434348 11391 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:14.344203 11422 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.448346  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.465853  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:14.510613  9859 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:14.468991  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.493323 11443 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:14.483563  9122 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:14.464516 10986 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.435038 11317 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.434628  9263 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:14.448002  9854 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.447624 11360 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:14.443853 11292 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:14.441533 11420 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.526628 11420 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:14.447283 11441 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 13:04:14.489714 11439 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:04:14.486377 11374 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:14.530019  9853 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:14.485864  9123 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.532532 11445 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:04:14.540934  9121 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:14.499533  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.500046  9825 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.500535  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.496632 11334 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:14.543355  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.544137 11102 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:14.544592  9238 leader_election.cc:400] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7.
I20260709 13:04:14.544932  9238 leader_election.cc:403] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:04:14.545933  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 45953.4us
I20260709 13:04:14.546844  9825 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.547552 11374 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:14.547854 11447 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.548472 11447 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.549284  9825 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.550323  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.550966 11447 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.551450 11367 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:14.503087 11426 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:04:14.551872 11367 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:14.507848 11442 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
W20260709 13:04:14.520500  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.519611  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.520668  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.532685  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:14.432889  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.533761  9119 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:14.531857  9117 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:14.535385  9122 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:14.505985 11339 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:14.520291 11444 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:04:14.505734 11391 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:14.494277  9089 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:14.503219 11423 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:04:14.505055 11424 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.560364  9857 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
W20260709 13:04:14.565877  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 66373.8us
I20260709 13:04:14.576349  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.517500 11443 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:14.569062  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 74848.7us
I20260709 13:04:14.560294  9123 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:14.578408  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.561798  9859 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:14.586252  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:14.587047  9121 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:14.586354  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.588173  9860 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:14.564209 11390 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:14.557616 11367 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:14.557478 11357 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:14.558768 11210 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:14.593762  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:14.564759 11378 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:14.570627 11391 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.578362 11443 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.539875 11317 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.578451 11356 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:14.601269  9123 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 2: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:14.584697 11256 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7.
I20260709 13:04:14.605533  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:14.605762  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 61342.7us
I20260709 13:04:14.608937  9860 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:14.564073 11387 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:04:14.606640  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.590878 11379 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:14.613548  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.592664 11390 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:14.622975  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.604911 11422 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:14.626741 11450 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:04:14.612361 10946 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.629539 11339 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.623677 11442 raft_consensus.cc:2764] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:04:14.622231 11390 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:14.670105  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.631835 11448 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:04:14.633494 11429 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:14.633490 11391 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.636646 10890 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:14.641269  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 317143us
I20260709 13:04:14.675952  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:14.676894  9161 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.641532  9853 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:04:14.643455  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.643967 11361 raft_consensus.cc:3055] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:14.645689  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:14.646905  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.648914  9855 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:14.606495 11323 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.657891  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.659291  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.607583 11356 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.660205  9121 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:14.661590  9861 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:14.621197 11357 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:14.666702  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:14.668365  9264 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:14.622763 11379 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:14.671277  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:14.671932  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:14.727451  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.727478  9271 raft_consensus.cc:2410] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:14.641526 11424 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:14.696574  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:14.682715  9853 pending_rounds.cc:85] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 5
I20260709 13:04:14.664276 11341 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.705856 11339 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:14.702157 11447 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:14.715212  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 55993.9us
W20260709 13:04:14.731211 11452 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:04:14.717885 11356 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.719487 11360 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:14.720420  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:14.723012  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.721479  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:14.723672  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.747803  9114 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:14.728425  9115 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:14.728853  9269 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:14.730633 11361 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:14.730930 11323 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:14.734292 11339 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.736591  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.715992 11459 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.709712 10946 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:14.713742 11421 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.742700  9192 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:14.763547  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:14.731761  9853 pending_rounds.cc:107] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.6
I20260709 13:04:14.724095  9860 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:14.750138  9121 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:04:14.768762  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.770745  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.750790  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.756623  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.758379  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.676518 11429 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:14.758878 11323 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:14.744599 11292 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:14.750375  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:14.771826  9190 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:14.776602  9122 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:14.756115 11424 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:14.776434  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:14.777498  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.777490  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.780616  9189 raft_consensus.cc:1813] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Handling vote request from an unknown peer ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.780699  9861 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:14.781131  9189 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:14.761942 11339 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.783059  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1790) took 1161 ms. Trace:
I20260709 13:04:14.783480  9195 rpcz_store.cc:276] 0709 13:04:13.621275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.621369 (+    94us) service_pool.cc:224] Handling call
0709 13:04:14.782859 (+1161490us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400398d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:14.782995 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818067200}
I20260709 13:04:14.775360  9855 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:14.784694  9825 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.786602  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.787456  9190 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:14.767645  9853 raft_consensus.cc:2917] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Aborting config change with OpId 4.6: Aborted: Op aborted by new leader
I20260709 13:04:14.759555 11420 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:14.790585  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:14.789708  9122 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:14.779882 11323 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.795598  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.795840  9860 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:14.796317  9190 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:14.797397  9190 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:14.805172  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.805871  9856 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:14.807089  9856 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:14.808974  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1916) took 1327 ms. Trace:
I20260709 13:04:14.793020  9189 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:14.794281  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:14.784897 11292 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.769009 10946 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:14.794983  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:14.810585  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:14.811276  9861 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:14.786273 11459 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:14.781324 11461 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:14.785746 11448 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:14.809371  9118 rpcz_store.cc:276] 0709 13:04:13.481465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.481580 (+   115us) service_pool.cc:224] Handling call
0709 13:04:13.481868 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.254787 (+772919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.808734 (+553947us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b5400254754. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:14.808898 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771451648}
I20260709 13:04:14.830789  9270 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:04:14.834414  9270 pending_rounds.cc:85] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 5
I20260709 13:04:14.834738  9270 pending_rounds.cc:107] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:04:14.834976  9270 raft_consensus.cc:2889] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.787542 11422 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:14.831627  9268 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:14.828842 11361 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.819545 11425 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:14.820397 11426 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:14.819473  9855 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:14.844132  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.849618 11367 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.852049  9196 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:14.852547  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1801) took 1011 ms. Trace:
I20260709 13:04:14.840095  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:14.868984  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:14.868296  9116 rpcz_store.cc:276] 0709 13:04:13.841595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.842106 (+   511us) service_pool.cc:224] Handling call
0709 13:04:13.901609 (+ 59503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:13.928578 (+ 26969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:13.928630 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:14.519467 (+590837us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800396520. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535c52a6 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a
0709 13:04:14.652026 (+132559us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b2c00000f48. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:14.652494 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:04:14.773379 (+120885us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:14.773419 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:14.773504 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:14.852366 (+ 78862us) raft_consensus.cc:1685] finished
0709 13:04:14.852384 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:14.852497 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19655,"raft.queue_time_us":11959,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":15161,"spinlock_wait_cycles":40261760,"thread_start_us":120518,"threads_started":1,"wal-append.queue_time_us":167105}
I20260709 13:04:14.856671  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:14.871912  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.872359  9267 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:14.872627  9265 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:14.858214  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:14.858628  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:14.784065 11424 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.873124  9272 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:14.874121  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:14.874693  9269 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:14.875171  9263 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:14.860154  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:14.814553  9853 raft_consensus.cc:2889] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.817011 11292 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.856516  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:14.840097 11317 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:14.819177 10946 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:14.793684 11395 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:04:14.866479  9861 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:14.865446 11211 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:04:14.860834 11360 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:04:14.825389 11429 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.908728  9859 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:15.223479  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:15.225467  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:15.226058  9119 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:14.935599  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:15.226675  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:15.226796  9268 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.012959  9272 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:14.995445  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 357) took 1026 ms. Trace:
I20260709 13:04:15.007251  9269 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:15.218916  9267 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:15.134116  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:15.213061  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:15.230561  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 370472us
W20260709 13:04:15.212410  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 273831us
I20260709 13:04:15.231726  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:15.227970  9120 rpcz_store.cc:276] 0709 13:04:13.969327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.030858 (+ 61531us) service_pool.cc:224] Handling call
0709 13:04:14.995270 (+964412us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5400253d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:14.995378 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31486336}
I20260709 13:04:15.234638  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:15.234855  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:15.236992  9161 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:14.995234  9089 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:15.237695  9119 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
W20260709 13:04:15.238370  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 243264us
I20260709 13:04:15.238967  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:15.240038  9194 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:15.240665  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:15.241269  9117 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:04:14.983889 11443 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.868301 11422 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.377s
I20260709 13:04:15.211457 11357 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:15.242669  9115 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:14.940495 11431 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:14.935999  9856 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:15.243297 11431 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:15.222569 11313 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.244174 11431 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:14.996068 11323 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:15.244892 11313 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:15.245424 11431 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:15.201529 11390 consensus_meta.cc:287] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.531s	user 0.005s	sys 0.003s
I20260709 13:04:15.245813  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:14.900388 11379 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:15.247004  9861 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:15.261193  9861 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:15.261709  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:15.262408  9861 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:15.263514  9861 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:15.246814  9118 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:15.229986  9853 pending_rounds.cc:107] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:04:15.247473 11390 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:15.183018 11430 consensus_meta.cc:287] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.005s
I20260709 13:04:14.927173 11391 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:15.000566 11467 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
W20260709 13:04:15.204519 11462 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 13:04:15.161659 11465 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:04:15.291838  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:15.292589 11356 consensus_meta.cc:287] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.000s
I20260709 13:04:15.292997  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:15.293284  9191 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.292995 11356 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:15.293416  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:15.293803  9190 raft_consensus.cc:2368] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3.
I20260709 13:04:15.294226  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:15.295185 11356 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:14.943655 11463 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:04:15.245362  9856 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEARNER]: Advancing to term 5
I20260709 13:04:15.296700  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:15.297233 11390 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:15.279383 11210 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:04:15.242091 11367 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:15.241007  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:15.252005 11360 consensus_queue.cc:1243] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:15.244298 11443 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:15.246749 11323 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:15.300329  9194 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:15.254582  9854 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:15.257644  9268 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:15.220358 11102 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.263922 11319 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:15.055115 11365 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:15.266021 11421 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:14.904029  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:15.197580 11420 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:15.262421  9198 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:15.268342  9089 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:15.278735 11317 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:15.266887  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:15.279570 11032 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:15.312739  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45938.4us
I20260709 13:04:15.280846  9862 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:15.245859 10946 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:15.286820  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:15.314595  9118 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:04:15.287634  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47789us
I20260709 13:04:15.288281 11384 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:04:15.318892  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.288961  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:15.319690  9119 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.289342 11341 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 13:04:15.063725 11468 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:04:15.295630  9191 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:15.295495  9195 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:15.189814 11389 raft_consensus.cc:3055] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:15.302743  9162 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:15.348616  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 45987us
I20260709 13:04:15.349814  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:14.865171 11427 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.413s
I20260709 13:04:15.309872 11421 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.357098  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:15.204923 11424 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:15.276786 11388 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.276961  9853 raft_consensus.cc:2889] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:14.870946 11334 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.434s
I20260709 13:04:15.274224  9117 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:15.313082  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 44812us
I20260709 13:04:15.314459 11210 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.337385  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:15.340565  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:15.134811 11339 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:15.199265 11425 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:15.265417  9122 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:04:15.423098  9122 pending_rounds.cc:85] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 4
I20260709 13:04:15.311412 11365 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:15.359551  9120 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:15.324220 10946 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.442430 10946 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:15.369593 11421 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:15.316685 11390 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.369690  9858 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:15.308811 11323 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 13:04:15.408893  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 51883.3us
I20260709 13:04:15.407506  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:15.335709  9119 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:15.464854  9119 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:15.417253  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:15.325652 11292 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:15.423467  9122 pending_rounds.cc:107] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:04:15.466591  9122 raft_consensus.cc:2889] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:15.424064  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:15.467885  9118 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:15.467005  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:15.364552  9861 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:15.366894 11318 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:15.468814  9272 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:15.468897 11318 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:15.469305 11318 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:15.469879  9272 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:15.470924  9272 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 6. Ops: []
I20260709 13:04:15.438196  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:15.348129 11389 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:15.309828 11102 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:15.386327 11317 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:15.368716  9271 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:15.453665  9192 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:15.368216 10890 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:15.405382 11420 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
W20260709 13:04:15.493783  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 86401.3us
I20260709 13:04:15.356261 11419 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
W20260709 13:04:15.489125  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 78532.8us
I20260709 13:04:15.488575  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:15.367926 11328 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.093s
W20260709 13:04:15.505523  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:15.506289  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.458313  9263 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:15.508867  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:15.462214 11425 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:15.492449 11210 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.516971  9858 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:15.492784 11292 raft_consensus.cc:697] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:15.458870 11319 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:04:15.504539 11374 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEARNER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:15.504830  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:15.540199  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:15.511983 11384 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.555424  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:15.520601  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:15.559732  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 219238us
I20260709 13:04:15.560555  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:15.560313  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43569.9us
I20260709 13:04:15.561739  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:15.562467  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:15.557642 11333 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:15.572839 11333 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:15.573081  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 64334.5us
I20260709 13:04:15.573343 11333 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:15.517578  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 719) took 1170 ms. Trace:
W20260709 13:04:15.571105 11468 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:15.473924 11317 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:15.537951 11356 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.577603  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:15.519927 11032 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:04:15.518551 11379 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:04:15.517993 11391 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:04:15.554229 11425 raft_consensus.cc:697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:15.569602  9266 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.569712 11376 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:15.570348 11313 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:15.473155 11389 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:15.582399 11474 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:04:15.580435 11480 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:15.604136  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:15.581256 11334 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:15.586639 11384 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.606819  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 44518.4us
I20260709 13:04:15.578559 11419 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:04:15.588351 11102 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:15.596930 11485 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:15.602599 11319 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:15.578375  9853 rpcz_store.cc:276] 0709 13:04:14.342994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.529606 (+186612us) service_pool.cc:224] Handling call
0709 13:04:14.529896 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:15.492961 (+963065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.512918 (+ 19957us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:15.621047  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:15.610720 11480 raft_consensus.cc:697] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:15.604377 10890 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:15.597251 11376 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:15.619765  9858 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:15.587920 11323 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:04:15.606925 11313 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:15.647008  9238 leader_election.cc:400] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3.
I20260709 13:04:15.673239  9238 leader_election.cc:403] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:04:15.620792 11032 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.616183 11334 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:15.633140 11443 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:15.627173 11391 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:15.610088 11487 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:15.626461 11361 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.618700 11328 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:15.624681 11365 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:15.631430 11382 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:04:15.637456 11485 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:15.625181 11481 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:04:15.617666 11384 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:15.625702 10946 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:15.624828 11491 consensus_queue.cc:1059] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:04:15.634672 11292 consensus_queue.cc:237] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:15.737861  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.702597  9194 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
W20260709 13:04:15.675803  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 99654.7us
W20260709 13:04:15.667627 11472 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:04:15.743542  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:15.693185  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 730) took 1193 ms. Trace:
I20260709 13:04:15.624758 11492 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:15.692761  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 71399.3us
I20260709 13:04:15.938289  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:15.965047  9858 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:15.966265  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:15.967131  9858 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:15.940503  9089 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:15.967809 11483 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.966594  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:15.974733  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:15.975545  9858 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.750223  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 708) took 1419 ms. Trace:
I20260709 13:04:15.670322 11339 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:15.775794 10890 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:15.740828 11437 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3.
I20260709 13:04:15.736080 11480 consensus_queue.cc:237] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:15.781626 11318 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:15.691936  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45055us
I20260709 13:04:15.667171 11389 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:15.751618 11399 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.939043 11484 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.949945 10946 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:15.952569 11384 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:15.953135 11391 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:15.725956 11376 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:15.860369  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.939395 11443 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:15.964098  9860 rpcz_store.cc:276] 0709 13:04:14.499439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.200725 (+701286us) service_pool.cc:224] Handling call
0709 13:04:15.201071 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.236381 (+ 35310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.236413 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.247662 (+ 11249us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.353445 (+105783us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:15.472435 (+118990us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.478242 (+  5807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.478339 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.692947 (+214608us) raft_consensus.cc:1685] finished
0709 13:04:15.692978 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:15.693123 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47627,"prepare.queue_time_us":118897,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":36709,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":9467,"thread_start_us":210994,"threads_started":2,"wal-append.queue_time_us":161866}
W20260709 13:04:15.725268  9855 consensus_meta.cc:287] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.847s	user 0.022s	sys 0.010s
I20260709 13:04:15.979387  9862 rpcz_store.cc:276] 0709 13:04:14.331263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.331524 (+   261us) service_pool.cc:224] Handling call
0709 13:04:14.339703 (+  8179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:15.313618 (+973915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.749962 (+436344us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b5400580f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:15.750151 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2163872640}
W20260709 13:04:15.736244 11475 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:04:15.815292 11211 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.742141  9856 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
W20260709 13:04:15.989389  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 432894us
W20260709 13:04:15.991770  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:15.778080 11469 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.747855 11497 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:15.726186 11419 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:15.998247 11494 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.010468 11421 consensus_meta.cc:287] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.000s
I20260709 13:04:15.692974 11032 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.013505  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1806) took 1146 ms. Trace:
I20260709 13:04:15.987870 11365 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.019616  9235 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:16.016829  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:15.695816 11319 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.307s
I20260709 13:04:15.989933 11328 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:04:16.032263  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.001626 11317 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:16.007412 11371 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:04:16.033547  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:16.017098 11442 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.027992  9856 pending_rounds.cc:85] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 4
I20260709 13:04:16.018698 11427 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.033085  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:15.005682  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:16.123731  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1779) took 1380 ms. Trace:
I20260709 13:04:16.135557  9114 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.135480  9192 rpcz_store.cc:276] 0709 13:04:14.742946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.768474 (+ 25528us) service_pool.cc:224] Handling call
0709 13:04:16.123451 (+1354977us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b54004173d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:16.123639 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1572924160}
I20260709 13:04:16.124276  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 765) took 1755 ms. Trace:
I20260709 13:04:16.125147  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:16.139915  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:16.141420  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:16.142479  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:16.154023  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:16.155014  9192 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.114049  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:16.137650  9857 rpcz_store.cc:276] 0709 13:04:14.368631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.847514 (+478883us) service_pool.cc:224] Handling call
0709 13:04:14.847847 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:14.932868 (+ 85021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.932911 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:14.932969 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:14.933011 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.123997 (+1190986us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b54005a9454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:16.124054 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.124200 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194985216}
I20260709 13:04:16.159495  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:16.160086  9857 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.000442 11483 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:15.677417 11425 consensus_queue.cc:237] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:15.988874 11492 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:16.020375  9855 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:16.022958 11210 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.081717  9860 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.032577 11376 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:16.015910 11341 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:04:16.016156 11479 raft_consensus.cc:3055] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:16.017277 11399 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:16.201325 11379 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.013s	sys 0.019s
I20260709 13:04:16.078763  9196 rpcz_store.cc:276] 0709 13:04:14.867296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.871592 (+  4296us) service_pool.cc:224] Handling call
0709 13:04:14.871886 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:14.878248 (+  6362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:14.878278 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.221474 (+343196us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.489663 (+268189us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:15.489698 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.489763 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:16.013172 (+523409us) raft_consensus.cc:1685] finished
0709 13:04:16.013203 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.013396 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6300,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7391,"thread_start_us":267772,"threads_started":1,"wal-append.queue_time_us":317456}
I20260709 13:04:16.165355  9115 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:16.081002  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 61474.3us
I20260709 13:04:16.096709 11339 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.106256 11303 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.263864 11379 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.014s	sys 0.019s
I20260709 13:04:16.273066 11339 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:16.273558 11318 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.167163  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:16.203568  9270 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:16.134361  9856 pending_rounds.cc:107] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:04:16.274636  9856 raft_consensus.cc:2889] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:16.275188 11507 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.275730  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 682) took 1705 ms. Trace:
I20260709 13:04:16.276183  9856 rpcz_store.cc:276] 0709 13:04:14.571033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.244963 (+673930us) service_pool.cc:224] Handling call
0709 13:04:15.245245 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:16.275463 (+1030218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.275664 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:16.277364  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:16.197520  9858 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:16.279099  9858 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:16.074234 11494 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.279217 11488 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:16.032066 11485 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:16.279814 11488 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.221211  9114 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:16.102471 11429 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.294538 11494 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:16.222239  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 234623us
W20260709 13:04:16.222913  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 281711us
I20260709 13:04:16.223062  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:16.298908 11374 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:16.178310  9857 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:16.077350 11497 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:16.232021  9197 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:16.299504 11374 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.301337 11497 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.301851 11497 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:16.302217 11374 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.302438 11379 raft_consensus.cc:3055] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:16.302850 11379 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:16.102595 11301 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:16.019316 11484 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.305224 11301 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:16.305626  9193 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:16.306663 11484 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.165869 11419 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.304401 11379 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.307623  9858 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:16.113727  9017 catalog_manager.cc:5697] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 4, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:04:16.307926 11379 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.235203  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 677) took 1680 ms. Trace:
I20260709 13:04:16.234681 11479 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:16.308750  9859 rpcz_store.cc:276] 0709 13:04:14.554472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.226165 (+671693us) service_pool.cc:224] Handling call
0709 13:04:15.226437 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:15.252730 (+ 26293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:15.252765 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:15.253273 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:04:15.366296 (+113023us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:15.609138 (+242842us) log.cc:844] Serialized 14 byte log entry
0709 13:04:15.609453 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:15.609577 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:15.738928 (+129351us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b180046ac80. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:16.224021 (+485093us) raft_consensus.cc:1685] finished
0709 13:04:16.234974 (+ 10953us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:16.235138 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25735,"prepare.queue_time_us":127478,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":27105,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":6587,"spinlock_wait_cycles":15280896,"thread_start_us":306009,"threads_started":2,"wal-append.queue_time_us":239552}
I20260709 13:04:16.200435  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:16.196920 11487 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.195678 11317 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.080176 11421 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.311631 11487 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.312966  9267 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:04:16.157239 11490 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:16.287173  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.313613 11487 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.315290 11492 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.321897 11328 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:16.325580 11492 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.326184 11492 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.327111 11483 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.329605 11399 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.287398  9860 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
W20260709 13:04:16.120401  9854 consensus_meta.cc:287] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.819s	user 0.013s	sys 0.008s
I20260709 13:04:16.067939 11391 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:16.069332 11469 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.344674  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:16.346419  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 325949us
I20260709 13:04:16.346807 11469 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.347354  9853 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:04:16.029992 11211 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.302263  9089 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:16.350725 11391 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.350924  9861 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:04:16.351992  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49803us
I20260709 13:04:16.352263 11102 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:16.353021 11102 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:16.353327  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:16.306308 11301 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.307353 11419 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.101544 11032 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.310014 11479 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.313550 11317 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.053102 11334 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.356604  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:16.080030 11443 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:16.280381 11485 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:16.373720 11429 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:16.284550  9192 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:16.206902 11442 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.232662  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 686) took 1601 ms. Trace:
I20260709 13:04:16.292203 11318 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:16.405048  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 48561.7us
I20260709 13:04:16.336958 11303 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:16.208570 11427 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:16.435300  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.376029 11489 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.412559 11429 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:16.431629  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.409938 11211 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.275774 11507 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.462036 11507 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.410697 11437 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.410364  9854 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:16.507148  9190 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.354110 11102 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:16.513236  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 55182.8us
I20260709 13:04:16.448812 11301 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.410518 11421 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.408777 11032 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:04:16.529340  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:16.534475  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:16.509590 11492 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.527086 11424 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.544631  9164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.014s	sys 0.045s
W20260709 13:04:16.528915 11390 consensus_meta.cc:287] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.067s	user 0.010s	sys 0.003s
I20260709 13:04:16.523954 11360 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.549356  9164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.014s	sys 0.045s
I20260709 13:04:16.422144  9855 rpcz_store.cc:276] 0709 13:04:14.624877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.794611 (+169734us) service_pool.cc:224] Handling call
0709 13:04:16.225892 (+1431281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97536}
I20260709 13:04:16.452310 11318 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:16.531262  9266 consensus_meta.cc:287] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.925s	user 0.008s	sys 0.004s
I20260709 13:04:16.464532 11489 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:16.555912 11333 consensus_meta.cc:287] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.982s	user 0.014s	sys 0.003s
I20260709 13:04:16.547406  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:16.550927 11313 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.011s	sys 0.025s
I20260709 13:04:16.571918  9270 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:16.562522  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:16.566030  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 51773.8us
I20260709 13:04:16.551251  9264 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:16.521793 11341 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:04:16.559437 11211 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.558279  9190 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:16.547609  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 706) took 2258 ms. Trace:
I20260709 13:04:16.585114  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:16.587213  9164 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:16.589109  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 395355us
I20260709 13:04:16.583549  9189 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:16.583616  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.565465 11390 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.575517  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:16.567166 11510 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:16.576143 11313 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.011s	sys 0.025s
I20260709 13:04:16.567720  9266 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:16.594571  9162 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:16.601866  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:16.560968 11424 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.604089  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:16.567482 11360 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.610317  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.606472  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:16.606657  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.612704  9119 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:16.614128  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.614939  9119 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:16.592222 11361 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.592896 11484 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:16.617074  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.611039  9122 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:16.591717 10890 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:16.620563  9118 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:16.577925 11333 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.568687 11429 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.596798 11490 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:16.628885  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:16.608543 11498 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:16.634500  9860 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:16.606280 11420 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:16.589692 11211 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.611871 11469 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:16.610203  9266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1589) took 1271 ms (client timeout 1521 ms). Trace:
I20260709 13:04:16.640069  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:16.625203 11488 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.629307 11361 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:16.633368 11359 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:04:16.638396 11442 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.626351 10890 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:16.633898 11399 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.657181  9860 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:16.633832 11490 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:16.659482  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 305271us
I20260709 13:04:16.631484 11419 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.585763  9854 rpcz_store.cc:276] 0709 13:04:14.281500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.299433 (+ 17933us) service_pool.cc:224] Handling call
0709 13:04:14.309884 (+ 10451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:15.253970 (+944086us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b5400078620. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f7656659a1d 00007f76566401a4 00007f76566c2447 00007f76566dafe7 00007f76566dc7de 00007f76566e6b49 00007f76566e7416 00007f76566e3d8f 00007f76566e383e 00007f765b6a8419
0709 13:04:16.515045 (+1261075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.533665 (+ 18620us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33054080}
I20260709 13:04:16.645010 11303 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.636976 11317 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:16.639910 11425 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:16.641981 11480 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:16.627506 11390 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:04:16.633373 11507 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:04:16.708587  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:16.630378 11492 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 13:04:16.670744  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 83619.1us
I20260709 13:04:16.710837  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:16.711529  9857 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:16.712563  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:16.646452  9266 rpcz_store.cc:269] 0709 13:04:15.338855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.419521 (+ 80666us) service_pool.cc:224] Handling call
0709 13:04:16.605666 (+1186145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:16.661635  9087 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:16.604009 11510 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:16.727210  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 65710.2us
I20260709 13:04:16.674962  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:16.687618  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 81221.4us
I20260709 13:04:16.727918  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:16.728242  9858 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:16.645246 11420 raft_consensus.cc:697] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:16.656493 11427 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.666654 10890 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:16.702911  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 67470.7us
I20260709 13:04:16.716257  9862 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:04:16.673578  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.734565  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:16.736501  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:16.691190 11341 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:16.748935  9164 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:16.744234  9860 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:16.741007  9119 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:16.767890  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.741295  9196 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:16.770143  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:16.750866  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:16.772189  9191 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:16.746659  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:16.746780 11313 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 13:04:16.752969 11514 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:04:16.741528  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:16.753896 11515 log.cc:927] Time spent T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:04:16.754743 11361 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.754676 11420 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.755897 11333 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.739216 11527 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.756260  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:16.757917 11510 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.760033 11513 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:16.760922 11210 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:16.761814  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:16.762606 11516 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:04:16.762809 11360 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.763342 11318 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.735829 11419 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 13:04:16.727047 11365 consensus_meta.cc:287] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.002s
I20260709 13:04:16.773927  9195 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
W20260709 13:04:16.733284 11506 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:04:16.672892 11399 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.759485 11211 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:16.726353 11317 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:16.784782  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 77383.8us
I20260709 13:04:16.788923  9198 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
W20260709 13:04:16.794392  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:16.795217  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 200717us
I20260709 13:04:16.746387  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:16.746299 11442 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:16.805884  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 57058.4us
W20260709 13:04:16.806411  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 65022.1us
I20260709 13:04:16.825541  9119 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:16.730815 10946 consensus_queue.cc:1059] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:04:16.668385 11483 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:04:16.736050 11491 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:04:16.763897 11517 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:04:16.784471 11427 raft_consensus.cc:2764] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:16.786827 11500 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:16.726135 11498 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:04:16.744829 11421 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:04:16.843329  9089 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:16.822376 11361 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:16.822229 11507 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.779553 11341 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.846729  9162 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:16.786057 11328 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:04:16.850816  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:16.859561  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:16.923130  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:16.862646  9195 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:04:16.831651  9854 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
W20260709 13:04:16.926071  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.837067 11512 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:16.839564 11365 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:16.838486 11530 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:16.833680 11529 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:16.848531 11376 consensus_meta.cc:287] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.000s
I20260709 13:04:16.916714 11391 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:16.899080  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:16.950891  9198 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:04:16.900250  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 57014.6us
I20260709 13:04:16.852758  9856 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:16.835260 11360 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:16.956945 11520 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:04:16.957047 11360 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:16.958443 11519 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:04:16.959538 11360 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:16.959790 11524 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:04:16.858202 11510 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:16.930634  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 83969.4us
I20260709 13:04:16.960729 11510 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.962783  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.952251 11317 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.963403 11510 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:16.874904 11500 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:16.964118  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.964329 11500 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:16.965277  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.874598 11333 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:04:16.864666 11303 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:04:16.953482  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:16.954427  9825 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:16.956185 11534 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:16.956048  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:16.956352 11427 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:16.940070  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:16.856871 11442 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:16.989398  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:16.970825  9116 raft_consensus.cc:2393] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:16.832862 11425 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:16.960227 11360 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:16.980026 11539 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:17.068152 11539 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:17.067229 11425 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.068610 11539 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:16.981266 11427 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.020818  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:16.942166 11512 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.009454  9856 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:17.038601  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:17.087540  9190 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:17.088505  9117 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:04:17.072499 11427 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:17.073304 11301 consensus_meta.cc:287] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
I20260709 13:04:17.072703 11388 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.097379 11388 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.026675 11492 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:17.041803  9194 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:16.919351 11361 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:16.949420 11530 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:17.054817 11442 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:17.042403 11494 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:17.066625 11493 raft_consensus.cc:3055] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:17.069564 11527 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.120157 11487 raft_consensus.cc:2955] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:17.073592  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 689) took 1375 ms. Trace:
I20260709 13:04:17.172520  9861 rpcz_store.cc:276] 0709 13:04:15.697824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:15.966243 (+268419us) service_pool.cc:224] Handling call
0709 13:04:17.073435 (+1107192us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540051b654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:17.073533 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865906944}
I20260709 13:04:17.098848 11399 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:17.109758 11492 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.079178  9197 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:16.945521 11529 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:17.100229 11526 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.337s	user 0.000s	sys 0.002s
W20260709 13:04:17.105795 11518 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.435s	user 0.000s	sys 0.001s
I20260709 13:04:17.048699 11510 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:17.113299  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:17.117112  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 398439us
I20260709 13:04:17.070210  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:17.130213 11493 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:17.070322  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:17.155391  9017 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 4, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 4 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:17.161285 11513 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:17.073719 11360 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.205832  9116 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:17.206480  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:17.207437  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:17.207782  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:17.247577  9268 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:17.235963  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:17.184679 11492 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:17.280658  9193 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 384) took 2997 ms (client timeout 1580 ms). Trace:
W20260709 13:04:17.286768  9193 rpcz_store.cc:269] 0709 13:04:14.283196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:14.997784 (+714588us) service_pool.cc:224] Handling call
0709 13:04:16.124952 (+1127168us) spinlock_profiling.cc:243] Waited 7.68 ms on lock 0x7b5800003098. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:17.280363 (+1155411us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b5400397e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:17.280536 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":400363008}
I20260709 13:04:17.147234 11494 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.201969 11529 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:17.324975  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1823) took 1177 ms. Trace:
I20260709 13:04:17.204794 11510 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:17.325547  9121 rpcz_store.cc:276] 0709 13:04:16.147293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.147841 (+   548us) service_pool.cc:224] Handling call
0709 13:04:16.202915 (+ 55074us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.203232 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.203257 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.203659 (+   402us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.570976 (+367317us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:16.571034 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.571109 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.101055 (+529946us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:17.324643 (+223588us) raft_consensus.cc:1685] finished
0709 13:04:17.324664 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:17.324895 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":580,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":23631,"spinlock_wait_cycles":75298560,"thread_start_us":301504,"threads_started":1,"wal-append.queue_time_us":396952}
I20260709 13:04:17.244251 11339 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:17.216391 11513 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:17.343897 11530 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:17.335513 11361 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:17.174403 11527 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.281342 11493 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.342052 11510 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:17.388718  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 537945us
I20260709 13:04:17.389837  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:17.391052  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:17.404454  9119 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:17.413349 11317 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:17.413945 11317 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.404676 11530 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:17.424232 11530 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:17.424711 11530 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.432288  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1791) took 1292 ms. Trace:
I20260709 13:04:17.432715  9194 rpcz_store.cc:276] 0709 13:04:16.141406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.152161 (+ 10755us) service_pool.cc:224] Handling call
0709 13:04:17.041563 (+889402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.103189 (+ 61626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.432065 (+328876us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54001d5e50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:17.432220 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2044797824}
I20260709 13:04:17.433964  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:17.399513 10890 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.624s	user 0.003s	sys 0.009s
I20260709 13:04:17.434585  9194 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:17.397938  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:17.465075  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 67228.2us
W20260709 13:04:17.380911 11469 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.004s	sys 0.008s
W20260709 13:04:17.486655  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 532331us
W20260709 13:04:17.500839  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 474092us
I20260709 13:04:17.433828 11500 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.458776 11317 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.513406 11301 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:17.471228 11339 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:17.513861  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:17.471362 11469 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.004s	sys 0.008s
W20260709 13:04:17.523865  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.472396 11494 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:17.530544  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:17.540951  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:17.553336  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:17.530817  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:17.614940  9122 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:17.471333 10890 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.519641 11493 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:17.624188  9117 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:04:17.616686  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 83311.5us
I20260709 13:04:17.618594  9118 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:17.642717  9089 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:17.521720 11339 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.650553  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:17.650882  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:17.657789  9116 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:17.526648 11437 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:17.536150 11399 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:17.546440 11390 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.554116 11541 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.683363  9089 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:17.546840  9854 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:17.689069  9196 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:17.634898 11360 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:17.628756 11388 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:17.526422 11301 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:17.640664 11498 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.652792  9263 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:17.662586 11339 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.653236 11032 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.677161 11494 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:17.730943  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46134.5us
I20260709 13:04:17.735153  9122 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:17.670078 11341 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:17.677029 11102 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:17.711931 11429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.026s	sys 0.022s
W20260709 13:04:17.738822  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 198467us
I20260709 13:04:17.651182 11538 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:04:17.745595  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:17.753008  9121 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:17.753175 11538 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:17.741865  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:17.736920 11490 consensus_meta.cc:287] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.046s	user 0.014s	sys 0.000s
I20260709 13:04:17.754491 11490 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:17.753821 11538 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:17.757529 11538 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.759243 11538 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.729281 11549 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:17.734941 11388 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:17.683943 11437 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:17.696344 11390 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:17.764146 11429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.026s	sys 0.023s
I20260709 13:04:17.695513 11399 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.797058 11301 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:17.797837  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:17.799512 11545 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:04:17.765021 11557 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:17.768832 11549 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:17.777837 11388 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:17.747109 11032 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:17.791865 11360 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:17.741523 11498 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:17.827551 11563 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:17.773692 11371 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:04:17.770241 11102 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:17.764717 11494 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.816998 11301 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:17.825532 11388 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:17.816663 11557 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:17.792047 11564 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:17.831382 11371 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:17.844583  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:17.871142  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:17.846283  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1824) took 1108 ms. Trace:
I20260709 13:04:17.837280 11500 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:17.849385 11102 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:17.849581 11317 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.854413  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:17.861253  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:17.882412  9857 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:17.883265  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:17.884483  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:17.885090  9857 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:17.861881  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:17.888161  9862 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:17.889213  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:17.890741  9862 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:17.828137 11563 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:17.862386  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:17.722247 11541 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.822433 11512 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:17.872400  9196 rpcz_store.cc:276] 0709 13:04:16.737457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.771185 (+ 33728us) service_pool.cc:224] Handling call
0709 13:04:16.771524 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.689643 (+918119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.846053 (+156410us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54004178d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:17.846226 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2109713152}
I20260709 13:04:17.851974 11341 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:17.880275 11559 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:17.896121  9860 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:17.897326  9859 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:17.875025 11371 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:17.899353  9862 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:04:17.833370 11360 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.878077  9192 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:17.912108  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:17.869043 11557 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.924010 11371 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:17.881369 11437 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:04:17.870379 11301 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:17.882972 11564 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.902101 10890 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:17.901721  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:17.990785  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 393) took 1042 ms. Trace:
I20260709 13:04:17.988834  9857 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:17.968102  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:17.912827  9263 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:18.001168  9192 rpcz_store.cc:276] 0709 13:04:16.948616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.877339 (+928723us) service_pool.cc:224] Handling call
0709 13:04:17.877703 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.925867 (+ 48164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.990725 (+ 64858us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:17.924124 11498 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:17.912604 11523 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 1.184s	user 0.000s	sys 0.003s
I20260709 13:04:17.913745 11390 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:17.927069 11500 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.026158  9862 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
W20260709 13:04:18.029434 11552 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:04:17.901490 11399 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:18.033113  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:18.052518  9268 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:17.999351 11437 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:17.925832 11341 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.071998 11564 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.074141 11480 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:18.075721 11480 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:18.076601 11480 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.084944 11494 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.054665 11500 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:18.098595 11547 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:04:18.099112  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1957) took 1769 ms. Trace:
I20260709 13:04:18.099538  9114 rpcz_store.cc:276] 0709 13:04:16.330178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.330531 (+   353us) service_pool.cc:224] Handling call
0709 13:04:16.330910 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.331312 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.331354 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.331970 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.569051 (+237081us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:16.569099 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.569182 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:17.850759 (+1281577us) spinlock_profiling.cc:243] Waited 36.3 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:18.098781 (+248022us) raft_consensus.cc:1685] finished
0709 13:04:18.098814 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.099031 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19191,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":71,"spinlock_wait_cycles":83475840,"thread_start_us":236648,"threads_started":1,"wal-append.queue_time_us":297398}
W20260709 13:04:18.087719 11510 consensus_meta.cc:287] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.672s	user 0.009s	sys 0.007s
I20260709 13:04:18.101858 11510 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.088882 11032 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:18.103807 11548 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 13:04:18.107375 11510 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:18.107842  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 779) took 1408 ms. Trace:
W20260709 13:04:18.108777 11492 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.786s	user 0.016s	sys 0.004s
I20260709 13:04:18.065359 11437 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:17.969208 11559 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.135735 11301 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:18.108294  9858 rpcz_store.cc:276] 0709 13:04:16.699888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.739281 (+ 39393us) service_pool.cc:224] Handling call
0709 13:04:16.739638 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.919919 (+180281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.919947 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.920426 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:04:17.516541 (+596115us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:17.516596 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.516669 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:18.107561 (+590892us) raft_consensus.cc:1685] finished
0709 13:04:18.107583 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.107766 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12817,"raft.queue_time_us":53897,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5155,"thread_start_us":432066,"threads_started":2,"wal-append.queue_time_us":595733}
I20260709 13:04:18.097959 11480 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:18.109558 11492 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.169693 11032 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.170297 11032 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } attrs { replace: true } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:18.170866 11530 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.000s
I20260709 13:04:18.137763 11317 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.173887 11360 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:18.177927 11328 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.180444 11328 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:18.173022  9828 connection.cc:256] RPC call timeout handler was delayed by -1.3575187760000631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.257s for a response
W20260709 13:04:18.182976  9828 connection.cc:256] RPC call timeout handler was delayed by -1.0482490580000103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2089s for a response
I20260709 13:04:18.183981  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:18.186012  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:18.187048  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:18.187948  9263 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:18.188953 11492 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.189272 11328 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:18.189630 11360 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:18.190279 11549 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:18.193394 11541 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:18.195777  9828 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:18.177780 11487 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.801s	user 0.013s	sys 0.000s
I20260709 13:04:18.201738  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:18.178128 11530 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.177616 11559 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.204555  9828 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:18.209651  9271 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.020s	sys 0.016s
I20260709 13:04:18.211182  9828 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:18.214154  9271 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.020s	sys 0.017s
I20260709 13:04:18.214051  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:18.225701  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:18.227178  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:18.200527 11541 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:18.203805 11360 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:18.233744  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:18.209182 11399 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:18.237450  9859 raft_consensus.cc:2955] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:18.222755 11469 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:18.196826 11510 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:04:18.215888 11361 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:18.241410 11553 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:04:18.229749  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:18.242738 11530 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.243211 11328 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:18.249533 11469 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.240674 11557 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:18.241811 11479 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:18.230172 11539 consensus_meta.cc:287] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.155s	user 0.014s	sys 0.000s
W20260709 13:04:18.281239 11427 consensus_meta.cc:287] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.192s	user 0.007s	sys 0.007s
I20260709 13:04:18.283592 11341 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.285463 11500 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:18.252295 11361 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.300686 11361 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:18.290624 11437 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:18.304052 11538 raft_consensus.cc:670] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:18.304265  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 740) took 1684 ms. Trace:
I20260709 13:04:18.305140  9855 rpcz_store.cc:276] 0709 13:04:16.620025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.621361 (+  1336us) service_pool.cc:224] Handling call
0709 13:04:16.662775 (+ 41414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.731992 (+ 69217us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:17.345809 (+613817us) log.cc:844] Serialized 14 byte log entry
0709 13:04:17.693166 (+347357us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:17.706843 (+ 13677us) log.cc:844] Serialized 26 byte log entry
0709 13:04:17.708893 (+  2050us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:17.709283 (+   390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:17.710147 (+   864us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:18.304033 (+593886us) raft_consensus.cc:1685] finished
0709 13:04:18.304054 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:18.304199 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30027,"prepare.queue_time_us":937729,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":22699,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":47066,"spinlock_wait_cycles":34749184,"thread_start_us":804873,"threads_started":2,"wal-append.queue_time_us":577246}
I20260709 13:04:18.307899 11399 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:18.308507 11399 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.310568 11399 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:18.311058 11399 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:18.311182 11493 consensus_meta.cc:287] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.658s	user 0.001s	sys 0.007s
I20260709 13:04:18.284715 11328 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:18.312093 11328 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:18.313190 11489 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.294378 11539 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.278247 11563 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:18.296221 11427 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.283900 11371 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.286881 11469 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:18.316473  9196 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:18.296566 11443 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:18.321820  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:18.337901  9826 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:18.347114  9189 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:18.305154 11538 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.337157  9263 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:04:18.340296 11339 consensus_meta.cc:287] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.591s	user 0.002s	sys 0.016s
I20260709 13:04:18.325976  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:18.311532 11399 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.340245 11500 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:18.340764 11510 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:18.363595  9086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.034s	sys 0.040s
W20260709 13:04:18.358870 11529 consensus_meta.cc:287] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.023s	user 0.013s	sys 0.000s
W20260709 13:04:18.367102  9086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.034s	sys 0.040s
I20260709 13:04:18.335700 11341 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:04:18.360004 11319 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.009s	sys 0.043s
I20260709 13:04:18.341691 11360 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 13:04:18.390991 11319 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.009s	sys 0.043s
W20260709 13:04:18.393520  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.374768 11500 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:18.361502 11538 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.363389 11339 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:18.413267 11420 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.000s	sys 0.024s
W20260709 13:04:18.408308 11388 consensus_meta.cc:287] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.001s
W20260709 13:04:18.439862  9089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.008s	sys 0.049s
W20260709 13:04:18.440949  9164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.017s	sys 0.018s
W20260709 13:04:18.441465  9164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.017s	sys 0.018s
W20260709 13:04:18.441108  9089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.008s	sys 0.049s
W20260709 13:04:18.443362  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 672595us
I20260709 13:04:18.445354  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:18.438575 11420 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.000s	sys 0.024s
I20260709 13:04:18.377930 11529 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:18.448276  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.010s	sys 0.011s
I20260709 13:04:18.411856 11032 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:18.453925  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.010s	sys 0.011s
I20260709 13:04:18.440809 11557 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:18.460970  9826 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.619s (SENT) [suppressed 9 similar messages]
I20260709 13:04:18.454270 11539 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.479173  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:18.479344  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:18.481669  9195 raft_consensus.cc:2368] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4.
I20260709 13:04:18.481578  9192 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:18.459327 11565 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:18.462234 11425 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.012s	sys 0.019s
W20260709 13:04:18.463845  9856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.009s	sys 0.014s
I20260709 13:04:18.453246 11388 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:18.500558  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:18.464026  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:18.505939  9161 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.017s	sys 0.012s
I20260709 13:04:18.519440  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:18.528787  9192 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
W20260709 13:04:18.528147  9161 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.017s	sys 0.012s
W20260709 13:04:18.532433  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 657659us
W20260709 13:04:18.498868 11425 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.012s	sys 0.020s
I20260709 13:04:18.489449 11563 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:18.488477 11360 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:18.506287  9237 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.012s	sys 0.017s
W20260709 13:04:18.542415  9237 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.012s	sys 0.017s
I20260709 13:04:18.508687  9164 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:18.501065  9856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.009s	sys 0.014s
I20260709 13:04:18.537012  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:18.544929  9198 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:18.538789  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.502161 11427 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.526233 11371 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.549223  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:18.562032  9860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.004s	sys 0.007s
I20260709 13:04:18.561264  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:18.553857  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:18.561877  9191 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:18.550563  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:18.564464  9268 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:18.555161  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:18.534569 11498 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:18.565769  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:18.567005  9268 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:18.569761  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:18.544478 11491 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.014s	sys 0.021s
I20260709 13:04:18.540699 11563 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:18.501812 11565 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:18.596196  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:18.581027  9268 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:18.547981 11559 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.545001 11437 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:18.545042 11489 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:18.547960 11492 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.545822 11399 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:18.546250 11317 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:18.602110  9195 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
W20260709 13:04:18.565378 11567 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:04:18.601758  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:18.554258 11494 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:18.554347 11341 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.602181  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:18.598656  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:18.603223  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:18.606838  9192 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:18.517719 11500 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:18.599809 11491 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.014s	sys 0.021s
W20260709 13:04:18.630990  9087 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.026s	sys 0.015s
I20260709 13:04:18.605273 11530 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:18.687942  9087 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.027s	sys 0.015s
W20260709 13:04:18.690382  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 955359us
I20260709 13:04:18.691443 11424 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:18.695068  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.685801 11341 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.698447 11341 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.698534 11361 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.699968  9858 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:18.655946 11569 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.701190 11569 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.702702 11565 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:18.562975  9860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.004s	sys 0.007s
I20260709 13:04:18.656586  9114 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:18.579855 11360 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:04:18.628710 11492 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:18.609885 11558 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.007s	sys 0.032s
I20260709 13:04:18.655802  9121 raft_consensus.cc:2410] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:18.628710 11489 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:18.873215  9859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.021s	sys 0.026s
W20260709 13:04:18.749042  9235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.026s	sys 0.030s
W20260709 13:04:19.061601  9235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.111s	user 0.033s	sys 0.048s
W20260709 13:04:19.055297  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.004s	sys 0.002s
I20260709 13:04:19.001930  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 708) took 1101 ms. Trace:
W20260709 13:04:19.062301  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.004s	sys 0.002s
I20260709 13:04:18.689867  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:18.752991  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48385.8us
W20260709 13:04:18.902503  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 368537us
W20260709 13:04:18.982753  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:19.059636  9826 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.000s	sys 0.012s
W20260709 13:04:18.793088  9238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.000s	sys 0.016s
W20260709 13:04:19.065526  9826 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.000s	sys 0.012s
W20260709 13:04:19.066128  9238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.002s	sys 0.029s
I20260709 13:04:18.654474 11371 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:18.562079 11522 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.868s	user 0.000s	sys 0.015s
I20260709 13:04:18.656049 11339 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:18.983938  9089 leader_election.cc:336] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.933s (SENT)
W20260709 13:04:18.982668  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 474060us
I20260709 13:04:19.099891  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:19.097033 11371 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:18.902699 11480 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.701s	user 0.007s	sys 0.004s
W20260709 13:04:19.059940  9859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.026s	sys 0.031s
W20260709 13:04:19.066278  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 605318us
I20260709 13:04:19.069104  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:19.062902  9860 rpcz_store.cc:276] 0709 13:04:17.905376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.924582 (+ 19206us) service_pool.cc:224] Handling call
0709 13:04:17.990883 (+ 66301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.991251 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:17.991277 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:17.991706 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 13:04:18.704030 (+712324us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:18.704085 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.704173 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.001641 (+297468us) raft_consensus.cc:1685] finished
0709 13:04:19.001667 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.001849 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":711975,"threads_started":1,"wal-append.queue_time_us":978681}
I20260709 13:04:19.021520  9017 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:18.699313 11341 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:19.076612 11425 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:18.610482 11437 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.059319 11569 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:19.080549 11575 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:04:18.654480 11493 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.177639  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.177870 11493 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:19.180640 11493 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:18.981962  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:19.097553  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:18.629581  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:18.605252 11563 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:19.181723  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1598) took 1974 ms (client timeout 2143 ms). Trace:
W20260709 13:04:19.182121  9272 rpcz_store.cc:269] 0709 13:04:17.207598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.207708 (+   110us) service_pool.cc:224] Handling call
0709 13:04:18.981755 (+1774047us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:19.181663 (+199908us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23950080}
I20260709 13:04:19.183254  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:19.183403 11469 consensus_meta.cc:287] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.841s	user 0.010s	sys 0.000s
W20260709 13:04:18.996857 11443 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.002s
I20260709 13:04:19.103802  9116 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:19.179373  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:19.213006  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.107650  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.231473  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1826) took 2493 ms. Trace:
W20260709 13:04:18.754370 11483 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.005s	sys 0.009s
I20260709 13:04:18.995138 11586 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:19.063254 11558 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.011s	sys 0.048s
I20260709 13:04:18.997469 11507 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:19.186075 11492 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.285157 11360 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.285326 11586 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.285853 11360 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.654882 11572 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:19.269855 11437 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.287554 11572 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:19.175868 11341 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:18.802464 11538 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.054162 11578 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.257310 11541 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:18.840082 11560 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.759625 11564 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:18.997530 11521 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 2.300s	user 0.004s	sys 0.000s
I20260709 13:04:18.690488 11530 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:19.296638 11530 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:19.297151 11572 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.289206 11541 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:19.297734 11560 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.298265 11541 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.939968 11388 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.113229  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:19.111457  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.303385  9162 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:18.973285 11529 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:18.761838 11328 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.072196 11317 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:18.583218 11498 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:18.609472 11559 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.079082 11513 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:18.707718 11420 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:18.605492 11427 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.499s
I20260709 13:04:19.285187 11507 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:18.950923 11399 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.156579  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:18.686558 11494 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.077162 11527 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.176780 11480 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:19.177234 11511 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 2.577s	user 0.000s	sys 0.014s
W20260709 13:04:19.173578  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 104535us
W20260709 13:04:19.194939  9270 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 513) took 1983 ms (client timeout 1899 ms). Trace:
I20260709 13:04:19.224462  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.340873  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:19.341017  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 514) took 2129 ms (client timeout 1639 ms). Trace:
I20260709 13:04:19.279387  9859 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:19.341563  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1954) took 3055 ms. Trace:
I20260709 13:04:19.184008  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1600) took 1973 ms. Trace:
I20260709 13:04:19.258378  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 388) took 2398 ms. Trace:
W20260709 13:04:19.344514  9270 rpcz_store.cc:269] 0709 13:04:17.211525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.194754 (+1983229us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:19.194869 (+   115us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:04:19.344882  9198 rpcz_store.cc:276] 0709 13:04:16.860126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.951611 (+ 91485us) service_pool.cc:224] Handling call
0709 13:04:17.027491 (+ 75880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.548029 (+1520538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.258036 (+710007us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54004173d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:19.258299 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3489494784}
I20260709 13:04:19.345465  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.346279  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:19.341584  9267 rpcz_store.cc:269] 0709 13:04:17.211804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.246560 (+ 34756us) service_pool.cc:224] Handling call
0709 13:04:19.224262 (+1977702us) spinlock_profiling.cc:243] Waited 5.88 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:19.340945 (+116683us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13527936}
I20260709 13:04:19.347033  9114 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:19.254419  9020 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42488 (request call id 238) took 8264 ms. Trace:
I20260709 13:04:19.226881  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.227612  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 378) took 2554 ms. Trace:
I20260709 13:04:19.215366  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 395) took 2217 ms. Trace:
I20260709 13:04:19.205873  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 392) took 2277 ms. Trace:
W20260709 13:04:19.210999  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.214319  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.248914  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 747) took 1015 ms. Trace:
I20260709 13:04:19.279271  9191 rpcz_store.cc:276] 0709 13:04:16.740774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.773032 (+ 32258us) service_pool.cc:224] Handling call
0709 13:04:16.773424 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.566722 (+1793298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.231142 (+664420us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400417dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:19.231381 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4112875264}
W20260709 13:04:19.282305  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 182155us
I20260709 13:04:19.237051  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.232482  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.281126  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:19.065500  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:19.217741 11598 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.192443 11391 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:19.172008 11588 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.446591 11598 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:19.283166 11483 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.146s	user 0.006s	sys 0.011s
I20260709 13:04:19.288407  9115 raft_consensus.cc:2410] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:19.447049  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.181751  9855 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:18.808331 11424 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.447682  9121 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:19.448101 11424 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.448709 11598 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:19.247287 11584 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 13:04:19.208480 11510 consensus_meta.cc:287] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.835s	user 0.010s	sys 0.000s
I20260709 13:04:19.295637  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:19.448815  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:19.449517  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:19.450364  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 155064us
I20260709 13:04:19.450516 11424 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:19.234387 11533 log.cc:927] Time spent T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 2.482s	user 0.004s	sys 0.000s
I20260709 13:04:19.451164  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:19.451526  9122 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:19.449695 11510 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:19.241467  9017 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 4 to 6, now has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 6 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:18.762285 11479 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.452662 11493 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:19.453096 11493 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:19.452948 11479 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.214382  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.453543 11493 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:19.454136  9266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1592) took 2248 ms (client timeout 2177 ms). Trace:
I20260709 13:04:19.185240 11489 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.188956  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 783) took 1680 ms. Trace:
I20260709 13:04:19.454819  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1953) took 3169 ms. Trace:
I20260709 13:04:19.455148  9856 rpcz_store.cc:276] 0709 13:04:17.508967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.510235 (+  1268us) service_pool.cc:224] Handling call
0709 13:04:17.514796 (+  4561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.521436 (+  6640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:17.526588 (+  5152us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:17.722785 (+196197us) log.cc:844] Serialized 26 byte log entry
0709 13:04:18.561633 (+838848us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:18.567013 (+  5380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.597039 (+ 30026us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.188710 (+591671us) raft_consensus.cc:1685] finished
0709 13:04:19.188731 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.188877 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1881,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":8192,"thread_start_us":763907,"threads_started":1,"wal-append.queue_time_us":1062269}
I20260709 13:04:19.455302  9120 rpcz_store.cc:276] 0709 13:04:16.285916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.332924 (+ 47008us) service_pool.cc:224] Handling call
0709 13:04:16.333290 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.582537 (+249247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.582587 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.593435 (+ 10848us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.715353 (+121918us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:16.715404 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.715473 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.233626 (+2518153us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:19.454514 (+220888us) raft_consensus.cc:1685] finished
0709 13:04:19.454538 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.454757 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16995,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"spinlock_wait_cycles":26173824,"thread_start_us":320407,"threads_started":2,"wal-append.queue_time_us":130690}
I20260709 13:04:19.204425 11490 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:19.203424 11323 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.289909  9853 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.458127 11323 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.245890  9110 tablet_service.cc:1558] Processing DeleteTablet for tablet 98d913b72e5b43e8b8f20fb99d0838c4 with delete_type TABLET_DATA_TOMBSTONED (TS ce51b61da22f40d4aa0204a6b1028f28 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43778
I20260709 13:04:19.293568 11361 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.299584 11595 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.460742 11323 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.460858 11595 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:19.461179 11493 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:19.454558  9266 rpcz_store.cc:269] 0709 13:04:17.205788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.212794 (+  7006us) service_pool.cc:224] Handling call
0709 13:04:19.214215 (+2001421us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:19.454063 (+239848us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50066048}
I20260709 13:04:19.463151 11595 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:19.301146 11437 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:19.312649 11599 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.158s	user 0.000s	sys 0.007s
I20260709 13:04:19.184062 11469 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.472831 11586 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:19.472834  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:19.473680  9862 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:04:19.474704 11559 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.474881 11469 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.464875 11568 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.476414 11568 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:19.340354 11480 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:19.477543 11479 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.173008  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:19.477941 11480 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.479013 11494 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.479352  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.479928 11512 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:19.480281  9263 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:19.481674  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:19.447130 11588 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:19.484313  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.300246 11569 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:19.299328 11541 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:19.102109 11522 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.408s	user 0.015s	sys 0.020s
I20260709 13:04:19.304774 11556 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.325929 11317 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.327766 11328 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.347535  9267 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:19.473727 11560 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.342346  9271 rpcz_store.cc:276] 0709 13:04:17.210361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.210493 (+   132us) service_pool.cc:224] Handling call
0709 13:04:17.236214 (+ 25721us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.522497 (+286283us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:17.699734 (+177237us) log.cc:844] Serialized 14 byte log entry
0709 13:04:18.217623 (+517889us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:18.218088 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:04:18.218223 (+   135us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:18.218258 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:18.218352 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.183806 (+965454us) raft_consensus.cc:1685] finished
0709 13:04:19.183829 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.183955 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":368105,"prepare.queue_time_us":445831,"prepare.run_cpu_time_us":5005,"prepare.run_wall_time_us":18934,"raft.queue_time_us":282341,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":15799,"spinlock_wait_cycles":14080,"thread_start_us":596229,"threads_started":2,"wal-append.queue_time_us":749428}
I20260709 13:04:19.436851  9272 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:19.495893 11301 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.489390  9266 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:19.496521 11301 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.497670  9271 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:04:19.440722  9189 rpcz_store.cc:276] 0709 13:04:16.928413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.615492 (+687079us) service_pool.cc:224] Handling call
0709 13:04:17.615767 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.347790 (+732023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.205615 (+857825us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b5400397e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:19.205813 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681489280}
I20260709 13:04:19.380441 11538 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.441591  9264 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:19.442970  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:19.498489 11538 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.342056  9123 rpcz_store.cc:276] 0709 13:04:16.286268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.330251 (+ 43983us) service_pool.cc:224] Handling call
0709 13:04:16.330621 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:16.331046 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:16.331086 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:16.331583 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:04:16.576896 (+245313us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:16.576940 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:16.577017 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:18.996938 (+2419921us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:19.328966 (+332028us) raft_consensus.cc:1685] finished
0709 13:04:19.328990 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.341490 (+ 12500us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19231,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":47353728,"thread_start_us":244865,"threads_started":1,"wal-append.queue_time_us":342467}
I20260709 13:04:19.498972 11301 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:19.499109  9264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1594) took 2292 ms (client timeout 1888 ms). Trace:
I20260709 13:04:19.443568  9196 rpcz_store.cc:276] 0709 13:04:16.998042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.916923 (+918881us) service_pool.cc:224] Handling call
0709 13:04:17.917505 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:18.330338 (+412833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.215070 (+884732us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b5400424fd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:19.215294 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917006208}
I20260709 13:04:19.289090 11578 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.499595  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.500548 11538 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:19.499665  9264 rpcz_store.cc:269] 0709 13:04:17.206377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.206543 (+   166us) service_pool.cc:224] Handling call
0709 13:04:19.236855 (+2030312us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:19.499035 (+262180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34006400}
W20260709 13:04:19.499027  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 56128.4us
I20260709 13:04:19.482393  9263 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:19.336521 11487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.021s	sys 0.010s
I20260709 13:04:19.448390  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.449165  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:19.296746 11564 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.452307  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.509256  9117 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:19.509405  9114 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:19.453375  9017 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:19.306809 11371 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:19.313256 11590 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.136s	user 0.003s	sys 0.003s
I20260709 13:04:19.452684  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:19.519559  9122 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:19.450524  9121 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:19.464241  9118 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:04:19.307111 11517 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:04:19.465741  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:19.440418  9119 rpcz_store.cc:276] 0709 13:04:16.673478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.123033 (+449555us) service_pool.cc:224] Handling call
0709 13:04:17.123442 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:17.404965 (+281523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.227284 (+1822319us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5400253d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:19.227507 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645492480}
I20260709 13:04:19.466552  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.524744  9116 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:19.506594 11491 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:19.469458  9120 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:19.470322  9265 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:19.471923  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:19.554255  9265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 515) took 2338 ms (client timeout 1876 ms). Trace:
W20260709 13:04:19.554860  9265 rpcz_store.cc:269] 0709 13:04:17.215925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.493220 (+277295us) service_pool.cc:224] Handling call
0709 13:04:19.214161 (+1720941us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:19.554184 (+340023us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35349760}
I20260709 13:04:19.472630 11489 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:19.497560 11493 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.565090  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.500684  9123 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:19.501332  9268 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.319144 11531 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:19.571771 11531 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:19.572345 11531 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:19.578604 11564 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.579339 11601 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:19.580082 11601 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:19.581562 11601 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.583699 11498 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.584267  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:19.587118  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:19.440256 11529 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:19.320286 11420 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.439183  9858 rpcz_store.cc:276] 0709 13:04:18.233221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.233398 (+   177us) service_pool.cc:224] Handling call
0709 13:04:19.248621 (+1015223us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b540051b654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:19.248811 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058188160}
I20260709 13:04:19.505192  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:19.505729  9271 pending_rounds.cc:85] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 6
I20260709 13:04:19.625068  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:19.625543  9271 pending_rounds.cc:107] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:04:19.625833  9271 raft_consensus.cc:2889] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:19.626006  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 120894us
W20260709 13:04:19.507277  9161 connection.cc:256] RPC call timeout handler was delayed by -1.1601375619999885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 13:04:19.510902  9017 catalog_manager.cc:5697] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 1 to 3, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:19.380849 11574 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.187s
W20260709 13:04:19.513618 11421 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.870s	user 0.002s	sys 0.018s
I20260709 13:04:19.630375  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:19.506843 11487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.021s	sys 0.015s
W20260709 13:04:19.467393  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 401982us
I20260709 13:04:19.526934  9117 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:19.499639 11301 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:19.657619  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.506628 11317 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.471516 11437 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.564384  9121 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:19.580127 11341 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:19.591820 11360 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:19.592315 11517 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:19.593911 11563 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.597344  9268 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:19.344034 11399 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.626704 11491 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:19.551307  9162 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:19.673704  9857 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:19.674266  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 123060us
W20260709 13:04:19.602656  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 136991us
I20260709 13:04:19.610234  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 748) took 1375 ms. Trace:
I20260709 13:04:19.671875 11479 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.676486  9862 rpcz_store.cc:276] 0709 13:04:18.234776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.234921 (+   145us) service_pool.cc:224] Handling call
0709 13:04:18.235245 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.474082 (+1238837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.609840 (+135758us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540009bad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:19.610149 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2847751552}
I20260709 13:04:19.677095  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.677037 11479 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.678303  9862 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:19.678925  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 757) took 1395 ms. Trace:
I20260709 13:04:19.679082 11479 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.611239  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.680060  9123 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:19.681998  9123 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:19.613062  9089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.021s	sys 0.039s
W20260709 13:04:19.684077  9089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.021s	sys 0.039s
W20260709 13:04:19.684643  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 700731us
I20260709 13:04:19.686575  9269 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:19.347388  9020 rpcz_store.cc:276] 0709 13:04:10.993855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:10.994078 (+   223us) service_pool.cc:224] Handling call
0709 13:04:19.254323 (+8260245us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:11.886804 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:11.980555 (+ 93751us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:11.994514 (+ 13959us) write_op.cc:435] Acquired schema lock
0709 13:04:11.994549 (+    35us) tablet.cc:662] Decoding operations
0709 13:04:12.355636 (+361087us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:12.379232 (+ 23596us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:12.382294 (+  3062us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:04:12.746376 (+364082us) tablet.cc:701] Row locks acquired
0709 13:04:12.769941 (+ 23565us) write_op.cc:260] PREPARE: finished
0709 13:04:12.878990 (+109049us) write_op.cc:270] Start()
0709 13:04:12.879249 (+   259us) write_op.cc:276] Timestamp: P: 1783602252815398 usec, L: 0
0709 13:04:12.879280 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:04:12.880144 (+   864us) log.cc:844] Serialized 4016 byte log entry
0709 13:04:13.982475 (+1102331us) op_driver.cc:464] REPLICATION: finished
0709 13:04:14.228096 (+245621us) write_op.cc:301] APPLY: starting
0709 13:04:14.228245 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:14.228593 (+   348us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:14.228767 (+   174us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:14.236268 (+  7501us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:14.236673 (+   405us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:04:14.236742 (+    69us) write_op.cc:312] APPLY: finished
0709 13:04:14.237216 (+   474us) log.cc:844] Serialized 77 byte log entry
0709 13:04:14.237461 (+   245us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:14.237631 (+   170us) write_op.cc:454] Released schema lock
0709 13:04:14.237714 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:14.321462 (+ 83748us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:14.326874 (+  5412us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30636,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":167346,"apply.run_cpu_time_us":4820,"apply.run_wall_time_us":44387,"num_ops":10,"prepare.queue_time_us":212399,"prepare.run_cpu_time_us":15920,"prepare.run_wall_time_us":1451819,"raft.queue_time_us":585625,"raft.run_cpu_time_us":12272,"raft.run_wall_time_us":234607,"replication_time_us":1069276,"thread_start_us":957698,"threads_started":4,"wal-append.queue_time_us":440149,"wal-append.run_cpu_time_us":122191,"wal-append.run_wall_time_us":1923346}]]}
I20260709 13:04:19.620818  9122 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:19.688921 11301 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:19.510681 11538 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:19.689400 11301 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:19.689837 11301 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:19.690151 11564 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:19.689724 11538 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.690722 11564 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:19.628721  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.692255  9196 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:19.692938 11564 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.693220  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.694103 11032 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:19.694362  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.694977  9196 raft_consensus.cc:1813] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Handling vote request from an unknown peer ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:19.629701  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.696066  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.696777  9194 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:19.629002  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.631048  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.699025  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:19.632807  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.700604  9857 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:19.635195  9017 catalog_manager.cc:5697] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 4, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:04:19.678876  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.637490  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:19.653702  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.655331  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:19.656627  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:19.705473  9192 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:19.505582 11569 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:19.657708  9115 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 1828) took 2903 ms (client timeout 1615 ms). Trace:
I20260709 13:04:19.519773  9855 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:19.658829  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:19.477494 11507 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.663594  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.733445  9198 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:19.731427  9196 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:19.652472 11421 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.018s	user 0.003s	sys 0.018s
I20260709 13:04:19.507373 11556 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:19.601559  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.652478 11333 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:19.688076 11437 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.694862  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:19.694653 11032 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:19.554303  9860 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:19.657475 11529 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:19.716369  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 78975.3us
I20260709 13:04:19.748359  9164 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:19.717298  9189 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:19.717882  9191 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:19.706192  9115 rpcz_store.cc:269] 0709 13:04:16.733248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:16.962537 (+229289us) service_pool.cc:224] Handling call
0709 13:04:19.624356 (+2661819us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b5400267354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:19.636321 (+ 11965us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5346876672}
I20260709 13:04:19.750633  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.751834  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.752533  9115 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:19.753700  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.754388  9115 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:19.702421  9017 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 3, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:19.660287 11389 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:04:19.755610  9017 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:19.679332 11588 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.767776  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:19.769361  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.732502  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:19.772424  9122 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:19.670095 11517 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:19.736397 11333 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:19.709149 11500 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.031s	sys 0.044s
I20260709 13:04:19.757534 11593 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:19.667487 11542 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:19.744346  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 789) took 1207 ms. Trace:
I20260709 13:04:19.788347  9860 rpcz_store.cc:276] 0709 13:04:18.537286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.436964 (+899678us) service_pool.cc:224] Handling call
0709 13:04:19.744280 (+307316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:19.778789  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:19.678509 11594 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.095s
I20260709 13:04:19.663460 11498 consensus_queue.cc:1059] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:04:19.679553  9862 rpcz_store.cc:276] 0709 13:04:18.286753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.677634 (+1390881us) service_pool.cc:224] Handling call
0709 13:04:19.677943 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.678690 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.678873 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:19.738322 11480 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:19.761472  9017 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 1 to 2, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:19.762959  9859 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:19.765008  9270 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:19.737298  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.772354  9825 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:19.768395  9271 raft_consensus.cc:2376] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3. Re-sending same reply.
I20260709 13:04:19.667296 11317 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.813074  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.856292  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.856956  9862 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:19.857818  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.858408  9862 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:19.859212  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.860322  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.847949  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.874799  9268 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:19.736836 11292 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:19.875885  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.777563  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.877715  9267 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:19.738968 11578 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.742482  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 785) took 1238 ms. Trace:
I20260709 13:04:19.741943 11564 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:19.653038 11512 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 13:04:19.786437  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.742482 11469 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:19.678314  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.852316 11032 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:19.862272  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 89977.7us
I20260709 13:04:19.917593  9825 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:19.862535 11588 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:19.827625  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 52560.4us
I20260709 13:04:19.786031 11333 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:19.864705 11399 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.866017  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:19.866744  9857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 794) took 1267 ms (client timeout 1577 ms). Trace:
I20260709 13:04:19.932785  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.868294  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.777134 11489 raft_consensus.cc:2764] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:04:19.844197 11600 log.cc:927] Time spent T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:04:19.939692  9121 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:19.872653  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 786) took 1370 ms. Trace:
I20260709 13:04:19.943454  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.853091  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 782) took 2359 ms. Trace:
I20260709 13:04:19.944197  9115 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:19.944376  9861 rpcz_store.cc:276] 0709 13:04:17.493936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.494074 (+   138us) service_pool.cc:224] Handling call
0709 13:04:17.494445 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:17.494953 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:17.652738 (+157785us) log.cc:844] Serialized 14 byte log entry
0709 13:04:19.218547 (+1565809us) spinlock_profiling.cc:243] Waited 7.77 ms on lock 0x7b180017b660. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:04:19.637487 (+418940us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.666995 (+ 29508us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.667154 (+   159us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:19.667193 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.667267 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.852753 (+185486us) raft_consensus.cc:1685] finished
0709 13:04:19.852861 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.853019 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2266,"prepare.queue_time_us":227820,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":45,"raft.queue_time_us":10587,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":15409,"spinlock_wait_cycles":17867136,"thread_start_us":553909,"threads_started":2,"wal-append.queue_time_us":2159295}
I20260709 13:04:19.847934 11334 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:19.739765  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.947167  9119 raft_consensus.cc:2410] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:04:19.737891  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 62165.7us
I20260709 13:04:19.868335  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:19.874272  9271 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:19.883272  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:19.948019  9272 raft_consensus.cc:1813] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Handling vote request from an unknown peer ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:19.881662  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.948719  9862 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
W20260709 13:04:19.948971  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:19.943147  9859 rpcz_store.cc:276] 0709 13:04:18.502279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.715405 (+1213126us) service_pool.cc:224] Handling call
0709 13:04:19.757773 (+ 42368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.872304 (+114531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.872581 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:19.886058  9123 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:19.943459  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:19.767843 11226 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:19.908216  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:19.775521 11494 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:19.787922 11500 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.031s	sys 0.045s
I20260709 13:04:19.821271 11542 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.812947 11593 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.921386  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:19.922410  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 382770us
I20260709 13:04:19.813832 11530 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:19.933497  9269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 511) took 2722 ms (client timeout 1619 ms). Trace:
I20260709 13:04:19.936865  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:19.937855  9857 rpcz_store.cc:269] 0709 13:04:18.599275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.112847 (+513572us) service_pool.cc:224] Handling call
0709 13:04:19.866557 (+753710us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400671e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:19.866682 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856359296}
I20260709 13:04:19.867012  9856 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:19.940311  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.779860 11560 consensus_queue.cc:1059] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.147s
I20260709 13:04:19.891844 11564 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:19.758920 11361 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:04:20.002487  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.956714  9191 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:04:19.956789  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:19.957567  9272 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:19.958223  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 50109us
I20260709 13:04:19.958390  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.005472  9119 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:19.959245  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:19.953763 11538 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.863068 11529 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.849376 11491 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:19.963280 11317 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.966977 11513 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:19.967285  9853 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.000s
I20260709 13:04:19.927776 11588 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:19.967694  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.968264  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.969156 11609 tablet_replica.cc:333] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:04:19.969617 11498 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:19.970571 11556 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:19.971076 11420 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.971349 11522 raft_consensus.cc:3055] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:19.972841 11480 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:19.883562  9855 rpcz_store.cc:276] 0709 13:04:18.498868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.448109 (+949241us) service_pool.cc:224] Handling call
0709 13:04:19.737486 (+289377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:19.974205 11569 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:19.926677 11586 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:19.975471  9263 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:19.976775  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1949) took 1396 ms. Trace:
I20260709 13:04:19.967694  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.977540  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.978956  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.980119 11563 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:19.927066 11469 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:19.981755  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 50635.1us
I20260709 13:04:19.984723 10890 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:19.985737  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.959950  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.983521  9162 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:19.994302  9267 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:19.995178  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:19.990469  9269 rpcz_store.cc:269] 0709 13:04:17.210999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:17.236839 (+ 25840us) service_pool.cc:224] Handling call
0709 13:04:19.232289 (+1995450us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:19.933299 (+701010us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400062a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:19.933415 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575192064}
I20260709 13:04:19.993770  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:19.760067 11507 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:20.002681  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:19.961298 11226 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:20.009634  9189 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:19.850190 11517 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:20.004622  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:19.989946 11494 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:20.008070  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:19.990548 11341 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:04:19.976779 11531 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:19.992372 11574 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
W20260709 13:04:20.526489  9120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.002s	sys 0.010s
I20260709 13:04:20.014310  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.852946  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:20.861450  9120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.002s	sys 0.014s
I20260709 13:04:20.850371  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1856) took 1342 ms. Trace:
I20260709 13:04:20.862167  9116 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:20.861513  9118 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:20.863087  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1860) took 1304 ms. Trace:
I20260709 13:04:20.863860  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 391) took 1167 ms. Trace:
I20260709 13:04:20.864399  9118 rpcz_store.cc:276] 0709 13:04:19.696320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.937351 (+241031us) service_pool.cc:224] Handling call
0709 13:04:20.863694 (+926343us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.863643  9120 rpcz_store.cc:276] 0709 13:04:19.558514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.563921 (+  5407us) service_pool.cc:224] Handling call
0709 13:04:19.564154 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.564340 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:19.882500 (+318160us) log.cc:844] Serialized 14 byte log entry
0709 13:04:20.862454 (+979954us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:20.862548 (+    94us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:20.862698 (+   150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.862798 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.863022 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":591010,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"thread_start_us":1297211,"threads_started":2,"wal-append.queue_time_us":878480}
I20260709 13:04:20.865617  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.866149  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.867096  9118 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:20.867321  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.874884  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:20.875794  9087 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.521s (SENT)
I20260709 13:04:20.876255  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.877655  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.879001  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.864109  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.880141  9116 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:20.881099  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:20.883394  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1864) took 1178 ms. Trace:
I20260709 13:04:20.884109  9116 rpcz_store.cc:276] 0709 13:04:19.705053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.882012 (+1176959us) service_pool.cc:224] Handling call
0709 13:04:20.882580 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.883023 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:20.883052 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:20.883111 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:20.883177 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.883233 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.883328 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:20.155277  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 450748us
W20260709 13:04:20.297633  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.887384  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:20.888517  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.889950  9161 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:20.892364  9119 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:20.892951  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 398) took 1114 ms. Trace:
W20260709 13:04:20.843870  9089 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.000s	sys 0.015s
W20260709 13:04:20.895835  9089 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.000s	sys 0.024s
W20260709 13:04:20.896462  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 889202us
I20260709 13:04:20.895862  9119 rpcz_store.cc:276] 0709 13:04:19.778987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.878619 (+1099632us) service_pool.cc:224] Handling call
0709 13:04:20.892866 (+ 14247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.898518  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:20.390043  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.916069  9119 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:20.916841  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1966) took 1156 ms. Trace:
I20260709 13:04:20.917209  9119 rpcz_store.cc:276] 0709 13:04:19.760842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.914855 (+1154013us) service_pool.cc:224] Handling call
0709 13:04:20.915460 (+   605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.916513 (+  1053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.916772 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.924158  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1968) took 1140 ms. Trace:
I20260709 13:04:20.871369  9121 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:20.014472  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:20.862228  9114 rpcz_store.cc:276] 0709 13:04:19.508151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.516527 (+  8376us) service_pool.cc:224] Handling call
0709 13:04:19.517060 (+   533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.517358 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.517395 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.556325 (+ 38930us) log.cc:844] Serialized 26 byte log entry
0709 13:04:20.002341 (+446016us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:20.002382 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.002472 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.849548 (+847076us) raft_consensus.cc:1685] finished
0709 13:04:20.849624 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.850068 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":338388,"threads_started":1,"wal-append.queue_time_us":1118172}
I20260709 13:04:20.895124  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1965) took 1147 ms. Trace:
I20260709 13:04:20.895051  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:20.898463  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:20.898986  9189 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:20.868883  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:20.899915  9192 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.000s	sys 0.013s
I20260709 13:04:20.867626  9117 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:20.901989  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:20.902170  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:20.903561  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1828) took 1272 ms. Trace:
I20260709 13:04:20.905566  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:20.910215 11635 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:20.910547 11634 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:04:20.911226  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 927666us
I20260709 13:04:20.911525  9087 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:20.928802  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 1963) took 1458 ms. Trace:
W20260709 13:04:20.695571  9264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.852s	user 0.012s	sys 0.036s
I20260709 13:04:20.930025  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:20.931424  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:20.935253  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1606) took 1166 ms. Trace:
I20260709 13:04:20.870222  9115 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:20.937016  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:20.937871  9270 rpcz_store.cc:276] 0709 13:04:18.580646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.381076 (+800430us) service_pool.cc:224] Handling call
0709 13:04:19.436299 (+ 55223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:19.859824 (+423525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.976550 (+116726us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b5400061b54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:19.976718 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":755217152}
I20260709 13:04:20.939370  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:20.940565  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:20.942888  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 934770us
W20260709 13:04:20.384200  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 215197us
W20260709 13:04:20.102960  9828 leader_election.cc:336] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.876s (SENT)
W20260709 13:04:20.944410  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:20.944798  9853 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:20.000463  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 749) took 1765 ms. Trace:
I20260709 13:04:20.946332  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:20.332980  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 753) took 2095 ms. Trace:
I20260709 13:04:20.991261  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:20.026021  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:20.949155  9862 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 4: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:20.478536  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:20.950354  9860 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:20.959955  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:20.964610 11226 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:20.447467 11301 consensus_meta.cc:287] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.757s	user 0.008s	sys 0.000s
I20260709 13:04:20.442178 11292 raft_consensus.cc:1064] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:04:20.965973 11317 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:20.746089 11319 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.010s	sys 0.001s
W20260709 13:04:20.659499 11333 consensus_meta.cc:287] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.720s	user 0.008s	sys 0.001s
I20260709 13:04:20.966784  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1963) took 1233 ms. Trace:
I20260709 13:04:20.969426  9119 rpcz_store.cc:276] 0709 13:04:19.784117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.918198 (+1134081us) service_pool.cc:224] Handling call
0709 13:04:20.923387 (+  5189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.923777 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:20.923805 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:20.923862 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:20.923902 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.923956 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.924082 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.969769  9118 rpcz_store.cc:276] 0709 13:04:19.747568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.893932 (+1146364us) service_pool.cc:224] Handling call
0709 13:04:20.894394 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.894783 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:20.894806 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:20.894859 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:20.894933 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.894990 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.895078 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.982862  9267 rpcz_store.cc:276] 0709 13:04:19.768805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.878480 (+109675us) service_pool.cc:224] Handling call
0709 13:04:19.878849 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.931026 (+1052177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.934677 (+  3651us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540028d7d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:20.935032 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264679296}
I20260709 13:04:20.017477 11542 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:20.990520  9191 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:20.990949  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 417) took 1029 ms. Trace:
I20260709 13:04:20.991067  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:20.991593  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 416) took 1032 ms. Trace:
W20260709 13:04:20.994105  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 82831.7us
I20260709 13:04:20.995025  9123 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:21.012677  9196 rpcz_store.cc:276] 0709 13:04:19.959497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.901495 (+941998us) service_pool.cc:224] Handling call
0709 13:04:20.991539 (+ 90044us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10496}
I20260709 13:04:20.999604  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 1980) took 1006 ms. Trace:
W20260709 13:04:20.621806 11601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.001s	sys 0.013s
I20260709 13:04:20.982899  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 393) took 1286 ms. Trace:
W20260709 13:04:20.985769  9192 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.000s	sys 0.014s
I20260709 13:04:20.989171  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 1975) took 1068 ms. Trace:
I20260709 13:04:20.990945  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:20.795346 11617 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:20.978929  9263 rpcz_store.cc:276] 0709 13:04:19.470041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.481360 (+ 11319us) service_pool.cc:224] Handling call
0709 13:04:20.928542 (+1447182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.008456  9116 rpcz_store.cc:276] 0709 13:04:19.733164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.885228 (+1152064us) service_pool.cc:224] Handling call
0709 13:04:20.885872 (+   644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:20.886501 (+   629us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:20.887152 (+   651us) log.cc:844] Serialized 14 byte log entry
0709 13:04:20.965863 (+ 78711us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:20.965923 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:20.965978 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.966043 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.966163 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":85450,"raft.queue_time_us":38686,"spinlock_wait_cycles":9620224,"thread_start_us":73779,"threads_started":2,"wal-append.queue_time_us":42042}
I20260709 13:04:21.011945  9198 rpcz_store.cc:276] 0709 13:04:19.959790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.905068 (+945278us) service_pool.cc:224] Handling call
0709 13:04:20.988727 (+ 83659us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:20.947670  9857 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
W20260709 13:04:20.789817 11623 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.054s	user 0.004s	sys 0.001s
I20260709 13:04:20.019719 11560 raft_consensus.cc:1064] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
W20260709 13:04:21.011113  9264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.163s	user 0.014s	sys 0.037s
I20260709 13:04:20.001969 11564 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:20.412735 11593 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:21.013976  9117 rpcz_store.cc:276] 0709 13:04:19.993191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.993480 (+   289us) service_pool.cc:224] Handling call
0709 13:04:20.985060 (+991580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.007941 11317 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:21.015312  9121 rpcz_store.cc:276] 0709 13:04:19.696949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.977291 (+280342us) service_pool.cc:224] Handling call
0709 13:04:20.973794 (+996503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.021034  9859 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:21.021162  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:20.379487 11530 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:21.021873 11522 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:21.036245  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.015506  9115 rpcz_store.cc:276] 0709 13:04:19.920362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.967462 (+ 47100us) service_pool.cc:224] Handling call
0709 13:04:20.984415 (+1016953us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:21.015261  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 116966us
I20260709 13:04:20.737970 11361 raft_consensus.cc:476] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 13:04:21.021164  9268 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:20.784859 11500 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:21.024302  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:20.774691 11568 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.028303  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.028615  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 411) took 1070 ms. Trace:
I20260709 13:04:21.028858 11483 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:21.029721  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1603) took 1261 ms. Trace:
I20260709 13:04:21.031867 11563 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:21.032778  9193 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:21.034286 11531 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:21.072872  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 181155us
I20260709 13:04:21.010962  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:21.038542  9114 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:21.039412  9270 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:21.031903  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:21.040279 11601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.054s	user 0.005s	sys 0.023s
I20260709 13:04:21.041024  9271 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:21.041704  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:21.042506 11560 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.044068  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 764) took 1082 ms. Trace:
I20260709 13:04:21.044271 11512 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:21.044816 11636 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.045620 11634 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:21.045861 11578 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:21.046425 11494 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:21.047003 11319 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.010s	sys 0.006s
I20260709 13:04:21.047114  9861 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:21.048133  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 726) took 1059 ms. Trace:
I20260709 13:04:21.045663  9825 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:21.037549  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 721) took 1082 ms. Trace:
I20260709 13:04:21.057436  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 763) took 1131 ms. Trace:
I20260709 13:04:21.119887  9862 rpcz_store.cc:276] 0709 13:04:19.942153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.000292 (+ 58139us) service_pool.cc:224] Handling call
0709 13:04:21.019708 (+1019416us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.119979  9859 rpcz_store.cc:276] 0709 13:04:19.926398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.381524 (+455126us) service_pool.cc:224] Handling call
0709 13:04:21.053254 (+671730us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.121471  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.121531  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.122310  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 771) took 1087 ms. Trace:
I20260709 13:04:21.122380  9859 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:21.100164 11323 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.122961  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 759) took 1220 ms. Trace:
I20260709 13:04:21.123311 11323 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:21.124531  9193 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1804) took 1495 ms (client timeout 1581 ms). Trace:
W20260709 13:04:21.124945  9193 rpcz_store.cc:269] 0709 13:04:19.628661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.628773 (+   112us) service_pool.cc:224] Handling call
0709 13:04:21.124296 (+1495523us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540010b654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:21.124431 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3066363776}
I20260709 13:04:21.056357 11483 raft_consensus.cc:697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:21.126663 11483 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.127341 11323 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.059981 11500 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:20.971503  9194 rpcz_store.cc:276] 0709 13:04:19.631074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.697303 (+ 66229us) service_pool.cc:224] Handling call
0709 13:04:19.697597 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.697992 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.698021 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.698518 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:04:20.902666 (+1204148us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:20.902821 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.902997 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:20.903075 (+    78us) raft_consensus.cc:1685] finished
0709 13:04:20.903095 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:20.903322 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60,"raft.queue_time_us":1975,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":36270,"thread_start_us":376182,"threads_started":1,"wal-append.queue_time_us":741425}
I20260709 13:04:21.060699  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 756) took 2782 ms. Trace:
I20260709 13:04:21.060861  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:21.061127 11498 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.061867 11630 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.100556 11636 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:21.063232  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:21.064157  9263 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:21.064898 11301 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:21.065994 11333 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.066553  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 395) took 1368 ms. Trace:
I20260709 13:04:21.067569  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.056929  9264 rpcz_store.cc:276] 0709 13:04:19.767990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.769036 (+  1046us) service_pool.cc:224] Handling call
0709 13:04:19.784233 (+ 15197us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.784613 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.784651 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.787240 (+  2589us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.029216 (+1241976us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:21.029293 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.029409 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.029462 (+    53us) raft_consensus.cc:1685] finished
0709 13:04:21.029472 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.029659 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4147,"raft.queue_time_us":730,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":25734,"thread_start_us":1176256,"threads_started":1,"wal-append.queue_time_us":910125}
I20260709 13:04:21.068183  9856 rpcz_store.cc:276] 0709 13:04:18.237274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.457372 (+1220098us) service_pool.cc:224] Handling call
0709 13:04:19.866684 (+409312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.992634 (+125950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.332655 (+340021us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400671e50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:20.332878 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940676992}
I20260709 13:04:21.069106 11542 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.071698  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1974) took 1171 ms. Trace:
I20260709 13:04:21.026113 11588 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.071833 11635 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.158346 11635 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:21.072535  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 1979) took 1068 ms. Trace:
I20260709 13:04:21.055044  9192 rpcz_store.cc:276] 0709 13:04:19.957775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.957916 (+   141us) service_pool.cc:224] Handling call
0709 13:04:19.979727 (+ 21811us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:20.000011 (+ 20284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:20.000045 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:20.000548 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:04:21.028017 (+1027469us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:21.028132 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.028260 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.028338 (+    78us) raft_consensus.cc:1685] finished
0709 13:04:21.028356 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.028559 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12697,"raft.queue_time_us":20853,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":47118,"thread_start_us":877720,"threads_started":1,"wal-append.queue_time_us":682567}
I20260709 13:04:21.039461  9265 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:21.076395  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1971) took 1257 ms. Trace:
I20260709 13:04:21.159874  9114 rpcz_store.cc:276] 0709 13:04:19.819469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.002224 (+1182755us) service_pool.cc:224] Handling call
0709 13:04:21.026898 (+ 24674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.076108 (+ 49210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.076343 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.076308 11522 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:21.077296  9017 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.012s	sys 0.013s
I20260709 13:04:21.077833  9121 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:21.031523  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 716) took 1101 ms. Trace:
I20260709 13:04:21.042778  9120 raft_consensus.cc:2955] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:21.090878  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:21.036187  9853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 791) took 2475 ms (client timeout 2654 ms). Trace:
I20260709 13:04:21.094477  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:21.093385  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 990434us
I20260709 13:04:21.093379 11292 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.099512  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.099963  9116 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:21.084316  9860 rpcz_store.cc:276] 0709 13:04:19.937538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.967450 (+ 29912us) service_pool.cc:224] Handling call
0709 13:04:21.016512 (+1049062us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.113386  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:21.047174 11564 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.114941 11586 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:21.115309 11643 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:21.034301 11317 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:21.119227  9857 rpcz_store.cc:276] 0709 13:04:19.988374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.994898 (+  6524us) service_pool.cc:224] Handling call
0709 13:04:21.036482 (+1041584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.181354  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1981) took 1165 ms. Trace:
I20260709 13:04:21.181618  9116 rpcz_store.cc:276] 0709 13:04:20.003605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.035920 (+1032315us) service_pool.cc:224] Handling call
0709 13:04:21.083905 (+ 47985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.168844 (+ 84939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.169016 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.122712  9862 rpcz_store.cc:276] 0709 13:04:20.035214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.121053 (+1085839us) service_pool.cc:224] Handling call
0709 13:04:21.122223 (+  1170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.062317  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.123349  9859 rpcz_store.cc:276] 0709 13:04:19.903018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.121128 (+1218110us) service_pool.cc:224] Handling call
0709 13:04:21.122913 (+  1785us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11648}
I20260709 13:04:21.137286  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 770) took 1129 ms. Trace:
W20260709 13:04:21.142766  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 101275us
I20260709 13:04:21.148543  9118 rpcz_store.cc:276] 0709 13:04:19.900676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.013760 (+1113084us) service_pool.cc:224] Handling call
0709 13:04:21.070865 (+ 57105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.071238 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.071275 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.071356 (+    81us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:21.071420 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.071487 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.071600 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.022337  9854 rpcz_store.cc:276] 0709 13:04:18.235092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:18.238147 (+  3055us) service_pool.cc:224] Handling call
0709 13:04:18.243983 (+  5836us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:18.609138 (+365155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:18.628094 (+ 18956us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:18.769661 (+141567us) log.cc:844] Serialized 26 byte log entry
0709 13:04:19.436605 (+666944us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:19.463549 (+ 26944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:19.475225 (+ 11676us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:19.981400 (+506175us) raft_consensus.cc:1685] finished
0709 13:04:19.988140 (+  6740us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:19.996446 (+  8306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6450,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":26152,"thread_start_us":686683,"threads_started":2,"wal-append.queue_time_us":820064,"wal-append.run_cpu_time_us":3370,"wal-append.run_wall_time_us":1356487}
I20260709 13:04:21.151679 11512 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:21.152324  9268 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:21.153330  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.154340 11542 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.155078  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:21.136394 11630 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.195626 11333 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:21.165952  9853 rpcz_store.cc:269] 0709 13:04:18.544222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.179051 (+634829us) service_pool.cc:224] Handling call
0709 13:04:21.016144 (+1837093us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.197077  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.197537  9856 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:21.197840  9853 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:21.198282  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 758) took 1295 ms. Trace:
I20260709 13:04:21.198586  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 766) took 1237 ms. Trace:
W20260709 13:04:21.191648  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.156167  9269 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:21.138588 11636 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.136204 11498 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:21.161540  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 124246us
W20260709 13:04:21.165729  9017 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.013s	sys 0.013s
I20260709 13:04:21.159564  9122 rpcz_store.cc:276] 0709 13:04:19.989357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.003603 (+ 14246us) service_pool.cc:224] Handling call
0709 13:04:21.053505 (+1049902us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:21.167234  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 121762us
I20260709 13:04:21.171725 11564 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:21.174008  9195 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:21.164060  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1982) took 1152 ms. Trace:
I20260709 13:04:21.176179  9161 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:21.176895 11635 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.176928  9828 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:21.177392  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:21.176175  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 151121us
I20260709 13:04:21.180012 11586 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.180501 11469 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.182621  9861 rpcz_store.cc:276] 0709 13:04:20.008008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.025657 (+ 17649us) service_pool.cc:224] Handling call
0709 13:04:21.133712 (+1108055us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.237171  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:21.237632  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.238086  9859 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:21.238390  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 761) took 1320 ms. Trace:
I20260709 13:04:21.238509  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 765) took 1297 ms. Trace:
I20260709 13:04:21.238826  9861 rpcz_store.cc:276] 0709 13:04:19.918437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.237372 (+1318935us) service_pool.cc:224] Handling call
0709 13:04:21.238336 (+   964us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.192693 11513 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:04:21.239602 11643 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.240337 11643 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.184402  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.241954  9263 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:21.193687 11361 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:21.242952 11361 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:21.242980 11643 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.195874 11630 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.243940  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.143021  9123 rpcz_store.cc:276] 0709 13:04:19.697634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.985429 (+287795us) service_pool.cc:224] Handling call
0709 13:04:21.066309 (+1080880us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54002f6cd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:21.066472 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261280512}
I20260709 13:04:21.192480 11500 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.184814 11494 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:21.208109  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.132000  9858 rpcz_store.cc:276] 0709 13:04:18.278486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.654542 (+1376056us) service_pool.cc:224] Handling call
0709 13:04:19.654950 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:19.655425 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:19.655471 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:19.662703 (+  7232us) log.cc:844] Serialized 26 byte log entry
0709 13:04:20.048730 (+386027us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:20.744142 (+695412us) log.cc:844] Serialized 14 byte log entry
0709 13:04:20.794711 (+ 50569us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:20.948809 (+154098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:20.948948 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:21.052834 (+103886us) raft_consensus.cc:1685] finished
0709 13:04:21.060229 (+  7395us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.060489 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":691275,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":57070,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":6361,"spinlock_wait_cycles":38791424,"thread_start_us":911815,"threads_started":3,"wal-append.queue_time_us":1070706}
I20260709 13:04:21.215915  9270 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 1.
I20260709 13:04:21.218173 11226 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:21.250252  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 73527.2us
W20260709 13:04:21.219491  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 55841.6us
I20260709 13:04:21.164114  9855 rpcz_store.cc:276] 0709 13:04:19.940887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.945534 (+1004647us) service_pool.cc:224] Handling call
0709 13:04:21.031424 (+ 85890us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.220250  9190 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:21.220049  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:21.220615  9860 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:21.222290 11301 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.225349  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:21.228089  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.212886  9117 rpcz_store.cc:276] 0709 13:04:20.011176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.048886 (+1037710us) service_pool.cc:224] Handling call
0709 13:04:21.088332 (+ 39446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.160608 (+ 72276us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:21.160659 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.163663 (+  3004us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:21.163728 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.163844 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.163991 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27614,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":12851,"thread_start_us":48148,"threads_started":1}
I20260709 13:04:21.228307  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.228788  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.229287 11507 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.230037 11522 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:21.256405  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:21.230736  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.260205  9857 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:21.260764  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 722) took 1318 ms. Trace:
I20260709 13:04:21.231225 11512 raft_consensus.cc:697] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:21.261556  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.261710  9826 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.619s (SENT)
I20260709 13:04:21.262360  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:21.262722 11640 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.199167  9853 rpcz_store.cc:276] 0709 13:04:19.960960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.196741 (+1235781us) service_pool.cc:224] Handling call
0709 13:04:21.198532 (+  1791us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.263521 11640 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.264457  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.265314  9853 raft_consensus.cc:2410] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:21.265724  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 724) took 1290 ms. Trace:
I20260709 13:04:21.265792 11640 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:21.266105  9853 rpcz_store.cc:276] 0709 13:04:19.975711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.264121 (+1288410us) service_pool.cc:224] Handling call
0709 13:04:21.265671 (+  1550us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:21.267274  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.261093  9857 rpcz_store.cc:276] 0709 13:04:19.942402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.185505 (+1243103us) service_pool.cc:224] Handling call
0709 13:04:21.260699 (+ 75194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.268579  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:21.269155  9858 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:21.269222  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 711) took 1782 ms. Trace:
I20260709 13:04:21.269694  9857 rpcz_store.cc:276] 0709 13:04:19.487044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.268286 (+1781242us) service_pool.cc:224] Handling call
0709 13:04:21.268671 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.268899 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.268923 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.268961 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:21.269011 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.269062 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.269172 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10496}
I20260709 13:04:21.271364  9857 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:21.243569 11361 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:21.254987  9828 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:21.255275  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.234812 11635 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.256405  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:21.272063  9862 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:21.274947  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 769) took 1280 ms. Trace:
I20260709 13:04:21.275305  9862 rpcz_store.cc:276] 0709 13:04:19.994364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.185606 (+1191242us) service_pool.cc:224] Handling call
0709 13:04:21.274894 (+ 89288us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:21.274544  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54649.4us
I20260709 13:04:21.256898 11226 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:21.257038 11643 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.257624  9197 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:21.257395 11292 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:21.238865  9859 rpcz_store.cc:276] 0709 13:04:19.940744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.185443 (+1244699us) service_pool.cc:224] Handling call
0709 13:04:21.238445 (+ 53002us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.266376 11640 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.272607  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 713) took 1785 ms. Trace:
I20260709 13:04:21.198675  9856 rpcz_store.cc:276] 0709 13:04:19.902649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.152938 (+1250289us) service_pool.cc:224] Handling call
0709 13:04:21.198106 (+ 45168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.250516 11494 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:21.284257  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59052.9us
I20260709 13:04:21.285192  9857 rpcz_store.cc:276] 0709 13:04:19.487654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.270685 (+1783031us) service_pool.cc:224] Handling call
0709 13:04:21.271077 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.271671 (+   594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.272572 (+   901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.286806  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:21.249897 11513 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.272837 11507 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:21.295722  9826 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.639s (SENT)
I20260709 13:04:21.282084 11635 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.305099  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1972) took 1459 ms. Trace:
I20260709 13:04:21.261235 11542 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.282379 11643 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:21.326087 11643 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.325958 11498 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.308034  9860 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:21.287487 11494 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:21.304885  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:21.308969  9266 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:21.309988  9857 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:21.304208 11507 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.311372  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 795) took 1603 ms. Trace:
I20260709 13:04:21.313544  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:21.314867 11531 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.286553 11634 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.287173 11469 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.301105 11301 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:04:21.316342  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:21.316828  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.316382 11333 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:21.262581 11512 consensus_queue.cc:237] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:21.318447 11317 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:04:21.320127  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.321502 11588 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.322178  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 787) took 2779 ms. Trace:
I20260709 13:04:21.323882  9861 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:21.325328 11527 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:21.325608  9858 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:21.308739 11642 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.328744  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 796) took 1620 ms. Trace:
W20260709 13:04:21.345475  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.345394  9857 rpcz_store.cc:276] 0709 13:04:19.708179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.288758 (+1580579us) service_pool.cc:224] Handling call
0709 13:04:21.309655 (+ 20897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.328503 (+ 18848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.328690 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.328605  9862 rpcz_store.cc:276] 0709 13:04:19.707808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.290076 (+1582268us) service_pool.cc:224] Handling call
0709 13:04:21.310781 (+ 20705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.311063 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.311087 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.311125 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:21.311188 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.311236 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.311325 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.331108  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 768) took 1336 ms. Trace:
I20260709 13:04:21.305683  9119 rpcz_store.cc:276] 0709 13:04:19.836543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.020505 (+1183962us) service_pool.cc:224] Handling call
0709 13:04:21.077437 (+ 56932us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b5800000c98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:04:21.077694 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.078052 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:21.142449 (+ 64397us) log.cc:844] Serialized 14 byte log entry
0709 13:04:21.288759 (+146310us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.288818 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:21.295413 (+  6595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.295499 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.295620 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26941,"prepare.queue_time_us":51878,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":2516,"raft.queue_time_us":12606,"spinlock_wait_cycles":6890752,"thread_start_us":139619,"threads_started":2}
W20260709 13:04:21.338560  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 50680.4us
W20260709 13:04:21.339593  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 84774.2us
W20260709 13:04:21.335705  9195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1832) took 1672 ms (client timeout 1522 ms). Trace:
I20260709 13:04:21.329530 11634 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:21.346522  9860 rpcz_store.cc:276] 0709 13:04:19.991585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.177029 (+1185444us) service_pool.cc:224] Handling call
0709 13:04:21.328137 (+151108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.308930 11631 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.328681 11500 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:21.330046 11531 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:21.348608  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.350369 11531 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.350687 11483 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:21.351259 11483 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:21.352097 11226 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.352413 11483 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:21.355983  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.352684 11226 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.297430 11513 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:21.346807  9195 rpcz_store.cc:269] 0709 13:04:19.663687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.667701 (+  4014us) service_pool.cc:224] Handling call
0709 13:04:21.335476 (+1667775us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b54001d4f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:21.335638 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3370474752}
I20260709 13:04:21.361059  9861 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:21.361512  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 719) took 1419 ms. Trace:
I20260709 13:04:21.361750  9861 rpcz_store.cc:276] 0709 13:04:19.941693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.239714 (+1298021us) service_pool.cc:224] Handling call
0709 13:04:21.361462 (+121748us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.345237 11642 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.348245 11634 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.362825 11642 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.362686 11333 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.349987 11631 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.369436 11631 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.337837 11469 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.372440  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 760) took 1469 ms. Trace:
I20260709 13:04:21.372833  9856 rpcz_store.cc:276] 0709 13:04:19.903357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.290243 (+1386886us) service_pool.cc:224] Handling call
0709 13:04:21.344024 (+ 53781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.372065 (+ 28041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.372105 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.372153 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:21.372213 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.372273 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.372385 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.371820 11588 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.344213 11636 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:21.779351  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.808655  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:21.469177  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:21.478427  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:21.472568  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:21.396934  9856 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:21.635878  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.814795  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 536550us
I20260709 13:04:21.429886  9858 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:04:21.396772 11595 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:21.496767  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:21.544294  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.825888  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:21.827029  9272 raft_consensus.cc:2368] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7.
W20260709 13:04:21.836763  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:21.453467  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 94926.6us
W20260709 13:04:21.837009  9120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.004s	sys 0.006s
W20260709 13:04:21.837424  9120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.004s	sys 0.006s
I20260709 13:04:21.826663  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 717) took 1885 ms. Trace:
I20260709 13:04:21.837858  9858 rpcz_store.cc:276] 0709 13:04:19.941218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.375947 (+1434729us) service_pool.cc:224] Handling call
0709 13:04:21.429397 (+ 53450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.826412 (+397015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.826609 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.421092  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.800105 11642 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:21.828933 11513 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:21.453003  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 136780us
W20260709 13:04:21.799796 11652 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
I20260709 13:04:21.829375  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:21.430367 11588 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:21.830439 11333 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:21.467999 11648 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:04:21.375181 11542 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:21.634805 11634 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:21.470201  9189 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 410) took 1513 ms (client timeout 1865 ms). Trace:
I20260709 13:04:21.476192  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 718) took 1534 ms. Trace:
W20260709 13:04:21.518918 11650 log.cc:927] Time spent T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Append to log took a long time: real 0.094s	user 0.001s	sys 0.003s
W20260709 13:04:21.814872  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 54730.1us
I20260709 13:04:21.833966  9269 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:21.834030  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1829) took 2202 ms. Trace:
I20260709 13:04:21.398538  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:21.860312  9857 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:21.860767  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:21.474664  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:21.826995  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 762) took 1906 ms. Trace:
I20260709 13:04:21.844080  9860 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:21.844118  9859 raft_consensus.cc:2368] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7.
W20260709 13:04:21.853638 11531 consensus_meta.cc:287] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 13:04:21.829317 11469 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:21.428800 11317 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.846989  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:21.848483  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.779026 11568 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:21.850210  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 1970) took 2031 ms. Trace:
I20260709 13:04:21.462867 11564 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:21.863492  9267 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:21.826637 11226 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:21.852336  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 538880us
I20260709 13:04:21.596686 11640 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:21.855880  9189 rpcz_store.cc:269] 0709 13:04:19.956342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.956486 (+   144us) service_pool.cc:224] Handling call
0709 13:04:21.470015 (+1513529us) spinlock_profiling.cc:243] Waited 5.3 ms on lock 0x7b540010c554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:21.470144 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12199808}
I20260709 13:04:21.835345 11595 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:04:21.845351  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.868745  9120 rpcz_store.cc:276] 0709 13:04:19.815145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:20.985467 (+1170322us) service_pool.cc:224] Handling call
0709 13:04:21.020434 (+ 34967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.042545 (+ 22111us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:21.327131 (+284586us) log.cc:844] Serialized 14 byte log entry
0709 13:04:21.845916 (+518785us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.845965 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:21.846018 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.846074 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.846178 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8211,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":6715,"thread_start_us":510092,"threads_started":1}
I20260709 13:04:21.861871  9856 rpcz_store.cc:276] 0709 13:04:19.920926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.396312 (+1475386us) service_pool.cc:224] Handling call
0709 13:04:21.396769 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.826758 (+429989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.826939 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.859283  9197 rpcz_store.cc:276] 0709 13:04:19.631387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.699793 (+ 68406us) service_pool.cc:224] Handling call
0709 13:04:19.761822 (+ 62029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.296696 (+1534874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.833722 (+537026us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400399254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:21.833958 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3439401344}
I20260709 13:04:21.863209  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:21.764717 11361 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.649363 11507 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:21.866890  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 388638us
I20260709 13:04:21.867509  9858 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:21.881970  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 720) took 1940 ms. Trace:
I20260709 13:04:21.882869  9858 rpcz_store.cc:276] 0709 13:04:19.941931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.846252 (+1904321us) service_pool.cc:224] Handling call
0709 13:04:21.864080 (+ 17828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.881751 (+ 17671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.881927 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:21.557853 11643 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.843874 11635 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:21.854732 11542 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:21.472111  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.534276 11494 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.853502 11654 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:21.870265 11564 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:21.654619 11630 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.871539  9268 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.876106  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:21.875919 11226 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:21.897608  9854 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:21.877264 11568 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.861382 11513 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:21.873566  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 414) took 1914 ms. Trace:
I20260709 13:04:21.882436 11563 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.899009 11563 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:21.899381 11563 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.899928 11568 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:21.900575  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:21.901027  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:21.901849 11317 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:21.902261 11317 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:21.902637 11317 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:21.902751  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:21.903605 11522 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:21.904079 11513 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:21.904690 11513 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:21.905226 11568 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:21.905743 11568 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:21.905622 11513 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.906694 11568 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.886216 11609 raft_consensus.cc:2243] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:21.895460 11564 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.344159  9853 rpcz_store.cc:276] 0709 13:04:18.528184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.266938 (+2738754us) service_pool.cc:224] Handling call
0709 13:04:21.267331 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.267724 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.273765 (+  6041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.275170 (+  1405us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:21.289345 (+ 14175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.297697 (+  8352us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.307080 (+  9383us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":388,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":151}
I20260709 13:04:21.909708 11361 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.908751 11609 raft_consensus.cc:2272] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:21.914986 11609 ts_tablet_manager.cc:1916] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:04:21.918469  9162 leader_election.cc:400] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7.
I20260709 13:04:21.918874  9162 leader_election.cc:403] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:04:21.919868 11498 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:21.920787 11226 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7.
W20260709 13:04:21.891526  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 487583us
I20260709 13:04:21.892295  9266 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
W20260709 13:04:21.889115 11631 consensus_meta.cc:287] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.001s
I20260709 13:04:21.893355 11542 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:21.858101  9855 rpcz_store.cc:276] 0709 13:04:19.941464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.474983 (+1533519us) service_pool.cc:224] Handling call
0709 13:04:21.475398 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.475779 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:21.475816 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:21.475888 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:21.475955 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:21.476017 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:21.476132 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:21.894266  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:21.891815  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:21.896754  9859 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:21.920002  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 45396.5us
I20260709 13:04:21.891527  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.892851  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:21.921370  9190 rpcz_store.cc:276] 0709 13:04:19.958855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.959022 (+   167us) service_pool.cc:224] Handling call
0709 13:04:21.868347 (+1909325us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400068bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:21.873515 (+  5168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2756264832}
I20260709 13:04:21.927145  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:21.866025 11531 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.936082  9857 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:21.955329  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 68332.1us
I20260709 13:04:21.960382  9828 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:21.962152 11361 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:21.962605 11361 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:21.963003 11361 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.963126 11654 consensus_queue.cc:1059] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:21.963953 11469 consensus_queue.cc:1059] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:21.964465 11654 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:21.964896 11469 raft_consensus.cc:3055] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:21.965023 11507 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.964555  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:21.965574 11588 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:21.981040 11588 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:21.982048 11588 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.982625 11634 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:21.983424 11588 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:21.965395 11654 raft_consensus.cc:3055] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:21.997823  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 123680us
I20260709 13:04:21.930959 11517 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:21.933658 11522 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:22.006569  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:21.995772 11500 consensus_meta.cc:287] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.001s
I20260709 13:04:22.009809  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:21.995007 11654 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:22.018826  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:21.932512 11631 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:21.965406 11469 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:22.015821  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.026355  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:22.030340  9263 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:22.031862  9264 raft_consensus.cc:2410] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:04:22.018158 11498 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:22.020606 11640 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:22.027107 11527 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:22.033651 11226 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:22.056602  9862 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:22.037642 11636 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.038977 11498 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:22.104235  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 254100us
I20260709 13:04:22.106005  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 723) took 2163 ms. Trace:
I20260709 13:04:22.106431  9857 rpcz_store.cc:276] 0709 13:04:19.954442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.864784 (+1910342us) service_pool.cc:224] Handling call
0709 13:04:21.891280 (+ 26496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.938711 (+ 47431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.105741 (+167030us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b5400511654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:22.105948 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98173312}
W20260709 13:04:22.109493  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 145064us
I20260709 13:04:22.109529 11667 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:22.039544  9266 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:22.111593  9828 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:22.115872 11667 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:22.116431 11667 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.116890 11667 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:22.123327  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 67734.5us
I20260709 13:04:22.054766 11226 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:22.125415  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 772) took 2024 ms. Trace:
I20260709 13:04:22.126919  9859 rpcz_store.cc:276] 0709 13:04:20.101493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.876646 (+1775153us) service_pool.cc:224] Handling call
0709 13:04:21.877007 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.931785 (+ 54778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.125165 (+193380us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b540009c754. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:22.125348 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44985216}
I20260709 13:04:22.112164 11498 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:22.143635 11469 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.145237 11469 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.154429  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.156565  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:22.158221  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.159960  9269 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:22.025822 11595 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.128783 11226 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.185904  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 725) took 2209 ms. Trace:
I20260709 13:04:22.189826  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:22.138141 11635 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:22.130199 11643 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:22.194290  9853 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:22.196846  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.199635  9164 leader_election.cc:400] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7.
I20260709 13:04:22.161592 11494 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.199899  9087 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:22.101152 11654 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.199183 11634 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:22.193439  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 63960.4us
I20260709 13:04:22.206571  9162 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:22.214165 11513 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:22.214812 11513 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.215260 11513 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:22.230422  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:22.232717  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:22.195094  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:22.243021  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 46741us
W20260709 13:04:22.245503  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 343963us
I20260709 13:04:22.204080 11531 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.203907 11630 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:22.236541  9087 raft_consensus.cc:2686] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:04:22.255335  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 802) took 1235 ms. Trace:
I20260709 13:04:22.255839  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.188601 11483 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:04:22.278724  9859 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
W20260709 13:04:22.279706  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 109303us
I20260709 13:04:22.278975  9858 rpcz_store.cc:276] 0709 13:04:21.029561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.889035 (+859474us) service_pool.cc:224] Handling call
0709 13:04:21.929316 (+ 40281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.021105 (+ 91789us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:22.113884 (+ 92779us) log.cc:844] Serialized 14 byte log entry
0709 13:04:22.254882 (+140998us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:22.254948 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:22.255004 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.255070 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.255233 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19894,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":7380,"thread_start_us":187058,"threads_started":2}
I20260709 13:04:22.235320 11595 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:22.288650  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 727) took 2297 ms. Trace:
I20260709 13:04:22.289147  9856 rpcz_store.cc:276] 0709 13:04:19.990901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.874991 (+1884090us) service_pool.cc:224] Handling call
0709 13:04:21.927334 (+ 52343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:22.004541 (+ 77207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:22.004571 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:22.031993 (+ 27422us) log.cc:844] Serialized 26 byte log entry
0709 13:04:22.254464 (+222471us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:22.254589 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.258805 (+  4216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:22.288411 (+ 29606us) raft_consensus.cc:1685] finished
0709 13:04:22.288429 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.288584 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24000,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":12035,"thread_start_us":126014,"threads_started":1,"wal-append.queue_time_us":233321}
W20260709 13:04:22.293746  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63434us
W20260709 13:04:22.294827  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 52505.3us
I20260709 13:04:22.252210 11507 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:22.248783  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:22.201054 11522 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:22.330561  9235 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.522s (SENT)
I20260709 13:04:22.307891  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:22.295858  9089 connection.cc:256] RPC call timeout handler was delayed by -1.812393534000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
W20260709 13:04:22.331954  9089 connection.cc:256] RPC call timeout handler was delayed by -1.7475935340000319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
W20260709 13:04:22.332288  9089 connection.cc:256] RPC call timeout handler was delayed by -1.712493534000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 13:04:22.332566  9089 connection.cc:256] RPC call timeout handler was delayed by -1.5432935339999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21250000000000002s for a response
W20260709 13:04:22.332818  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0671935340000118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2654s for a response
W20260709 13:04:22.333105  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0653935340000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2656s for a response
W20260709 13:04:22.308871  9854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 767) took 2322 ms (client timeout 2864 ms). Trace:
W20260709 13:04:22.333634  9854 rpcz_store.cc:269] 0709 13:04:19.987095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.227950 (+1240855us) service_pool.cc:224] Handling call
0709 13:04:22.308702 (+1080752us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400660b54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:22.308805 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1459471616}
I20260709 13:04:22.309844  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 537) took 1386 ms. Trace:
I20260709 13:04:22.334472  9264 rpcz_store.cc:276] 0709 13:04:20.923462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.149331 (+225869us) service_pool.cc:224] Handling call
0709 13:04:22.309621 (+1160290us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54000ebfd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:22.309780 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1918088960}
I20260709 13:04:22.314275 11642 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:22.317207  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.317539  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:22.323801  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:22.324678  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.324882  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 212766us
I20260709 13:04:22.293068 11631 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.328166  9855 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:22.328663  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:22.343791  9858 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:22.291920 11317 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.300535 11654 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.340572  9268 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:22.344383  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:22.304191 11483 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:22.347036  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.347306  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:22.345575  9235 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.577s (SENT)
I20260709 13:04:22.349807  9853 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:22.242172 11563 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.350631  9263 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:22.355808  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:22.335224 11634 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.355083  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:22.247767 11542 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.186455  9860 rpcz_store.cc:276] 0709 13:04:19.976249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.862264 (+1886015us) service_pool.cc:224] Handling call
0709 13:04:21.880872 (+ 18608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:21.909143 (+ 28271us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:21.928784 (+ 19641us) log.cc:844] Serialized 14 byte log entry
0709 13:04:22.164152 (+235368us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:22.164210 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:22.185592 (+ 21382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:22.185694 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:22.185848 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12887,"prepare.queue_time_us":101752,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":5939,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":2503,"thread_start_us":173028,"threads_started":2}
W20260709 13:04:22.357611  9826 connection.cc:256] RPC call timeout handler was delayed by -1.0705126739999855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
I20260709 13:04:22.355121  9859 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:22.354689 11568 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:22.362435 11568 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:22.365837 11568 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:22.366572  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.368016  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.368575  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.369146  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.369774  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.370289  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:22.370577 11662 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:04:22.370925  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:22.372579  9235 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.766s (SENT)
I20260709 13:04:22.372786  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:22.373129  9235 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:22.374264 11542 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:22.375195  9861 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 6: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:22.375974 11500 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:22.377231  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 732) took 1055 ms. Trace:
I20260709 13:04:22.377679  9861 rpcz_store.cc:276] 0709 13:04:21.321416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:21.362227 (+ 40811us) service_pool.cc:224] Handling call
0709 13:04:22.376987 (+1014760us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400524254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:22.377163 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1213860096}
I20260709 13:04:22.356813  9862 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:22.378757  9263 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:22.380129  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.359922  9269 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:22.381400  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:22.302548 11531 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:04:22.391788  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:22.387449  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:22.392470  9854 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:22.387506  9162 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:22.346997  9266 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.339668 11507 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:22.396638  9264 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:22.386013  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:22.381073 11642 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.384460 11634 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.389066 11317 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:22.390054  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.390973  9826 leader_election.cc:336] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.899s (SENT)
I20260709 13:04:22.371568  9272 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:22.359120 11483 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:22.401571  9828 connection.cc:256] RPC call timeout handler was delayed by -1.6335589600000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
I20260709 13:04:22.406826 10890 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:22.407400 10890 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:22.407033 11507 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:22.408318 10890 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:22.418247  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:22.422032  9235 leader_election.cc:336] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.964s (SENT)
I20260709 13:04:22.423811  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:22.420603  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:22.411949 11483 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:22.431540  9115 raft_consensus.cc:2393] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:22.423307  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:22.432583  9120 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:22.434490  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:22.415244 11542 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.437448  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:22.437992  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.439407  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:22.441610  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:22.442587  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:22.443636  9122 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:22.446240  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:22.447413  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:22.449512  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.452716  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.453923  9271 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:04:22.457079  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 66141us
I20260709 13:04:22.466212 11333 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:22.472740 11563 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.474083  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.483814  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:22.475464  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.489061  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.491297  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:22.491294  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.496491  9857 raft_consensus.cc:2393] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:22.494017  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:22.497272  9271 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:22.426330 11642 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:22.488049 11361 consensus_meta.cc:287] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.001s
I20260709 13:04:22.496048  9117 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:22.499084  9860 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:22.502386  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:22.504609  9123 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.462512 11631 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:22.490935 11667 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:22.509044  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 84101.7us
I20260709 13:04:22.505553 11679 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:22.453428 11542 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:22.508013 11588 consensus_meta.cc:287] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.001s
I20260709 13:04:22.511348  9857 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:04:22.512538  9862 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:22.456459 11226 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.515228 11667 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:22.516660  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:22.510857  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.510246  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:22.541862  9859 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:04:22.542755  9263 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:22.548061  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:22.560307  9235 leader_election.cc:336] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 2.085s (SENT)
I20260709 13:04:22.560854  9235 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:22.560913  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54157.6us
I20260709 13:04:22.562960  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:22.516335 11679 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:22.590762 11568 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:22.555451 11498 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:22.507829 11361 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.593268  9123 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:22.596524 11595 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:22.601150  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48375.5us
W20260709 13:04:22.601210  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63008.5us
I20260709 13:04:22.509814 11513 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.606711  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:22.605937  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:22.591859 11667 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:22.612442  9862 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:04:22.616609  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:22.594319 11679 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.607852 11588 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:22.593093 11683 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.191s
I20260709 13:04:22.615415 11494 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:22.626909 11494 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:22.627343 11494 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:22.615216 11563 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:22.628078 11563 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:22.628504 11563 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:22.649151 11542 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:22.650126 11690 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.650755 11690 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.652995 11690 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:22.653662 11673 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:04:22.657784 11483 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.661471  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:22.662645  9115 raft_consensus.cc:2410] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:22.663700  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:22.664359  9115 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:22.675118  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:22.675592  9860 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:04:22.675988  9860 pending_rounds.cc:85] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 4
I20260709 13:04:22.675846  9118 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:22.676224  9860 pending_rounds.cc:107] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:04:22.676563  9860 raft_consensus.cc:2889] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:22.677711  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:22.677951  9118 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:04:22.678956  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 116105us
I20260709 13:04:22.679179  9122 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:22.679803  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:22.680576  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:22.681226  9120 raft_consensus.cc:2368] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4.
W20260709 13:04:22.682330  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 70435.3us
I20260709 13:04:22.683056  9087 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:22.621002 11634 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.694028  9116 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:04:22.687768  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:22.694700  9121 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:22.691576  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 131323us
I20260709 13:04:22.694854  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:22.695634  9117 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.697048  9266 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:22.612695 11568 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.700870 11226 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.701702 11642 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.702306  9120 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:04:22.702314 11640 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:22.703389 11640 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.703703 11640 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:22.706701 11498 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:22.667922 11636 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:22.707882 11636 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.708263 11636 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:22.712417 11683 raft_consensus.cc:3055] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:22.712971 11683 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:22.713219  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:22.713801 11683 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.701713 11568 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:22.712327  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 80234.6us
I20260709 13:04:22.715214 11683 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:22.716195 11588 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:22.716394  9235 leader_election.cc:336] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.572s (SENT)
I20260709 13:04:22.717204  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:22.722739  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:22.723488  9859 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:22.724445  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:22.726563 11595 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.727185 11595 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.728103 11361 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:22.616987 11635 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:04:22.717758 11634 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:22.718494 11690 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:22.687592 11522 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:22.720644  9825 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:22.729784  9854 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:22.755676  9861 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:04:22.756242  9117 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:22.717631  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:22.734836  9114 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:22.734491  9115 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:22.686786 11542 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.720331  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:22.686964 11609 ts_tablet_manager.cc:1929] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:04:22.750974  9121 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:22.751777  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.760448  9164 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:22.763120 11513 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.752596  9856 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:04:22.758649  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 75708.2us
I20260709 13:04:22.748787 11531 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:22.778367  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 60825.8us
I20260709 13:04:22.776485  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:22.779184  9164 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:22.755455 11483 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:22.786522  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 65986.6us
I20260709 13:04:22.787403  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:22.788451  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:22.781884 10890 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:04:22.825481  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 47131.8us
W20260709 13:04:22.805557  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:22.793102  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 359389us
I20260709 13:04:22.826385  9826 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:22.829309  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:22.831168  9121 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:22.790030  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:22.832024  9114 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:22.841123 11654 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:22.841897 11630 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:22.776860 11542 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.845592 11542 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:22.775501 11588 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:22.776571 11609 log.cc:1199] T 98d913b72e5b43e8b8f20fb99d0838c4 P ce51b61da22f40d4aa0204a6b1028f28: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/wals/98d913b72e5b43e8b8f20fb99d0838c4
I20260709 13:04:22.846534  9860 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:22.847563  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:22.850603  9859 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:22.778136 11531 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.775975 11634 consensus_queue.cc:1243] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:22.878311  9087 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:22.790588 11690 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:22.859982 11568 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:22.783139 11513 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:22.790302 11361 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:22.854424 11636 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.829259 11586 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:22.878199 11531 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:22.825322 11595 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:22.912112  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59493.6us
W20260709 13:04:22.913075  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.885627  9017 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36610 (request call id 324) took 9525 ms. Trace:
I20260709 13:04:22.922386  9017 rpcz_store.cc:276] 0709 13:04:13.349967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.350247 (+   280us) service_pool.cc:224] Handling call
0709 13:04:22.875645 (+9525398us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:21.059343 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:21.118484 (+ 59141us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:21.118532 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:04:21.118564 (+    32us) tablet.cc:662] Decoding operations
0709 13:04:21.171337 (+ 52773us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:21.171423 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:21.171456 (+    33us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:04:21.172513 (+  1057us) tablet.cc:701] Row locks acquired
0709 13:04:21.172563 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:04:21.172770 (+   207us) write_op.cc:270] Start()
0709 13:04:21.172986 (+   216us) write_op.cc:276] Timestamp: P: 1783602261172700 usec, L: 0
0709 13:04:21.173003 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:04:21.174336 (+  1333us) log.cc:844] Serialized 4340 byte log entry
0709 13:04:21.511917 (+337581us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:22.201880 (+689963us) op_driver.cc:464] REPLICATION: finished
0709 13:04:22.376468 (+174588us) write_op.cc:301] APPLY: starting
0709 13:04:22.376583 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:22.376839 (+   256us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:22.376883 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:22.475039 (+ 98156us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:22.475449 (+   410us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:04:22.475504 (+    55us) write_op.cc:312] APPLY: finished
0709 13:04:22.508920 (+ 33416us) log.cc:844] Serialized 77 byte log entry
0709 13:04:22.536419 (+ 27499us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.548310 (+ 11891us) write_op.cc:454] Released schema lock
0709 13:04:22.556401 (+  8091us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:22.556962 (+   561us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:22.556993 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3647788,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":168864,"apply.run_cpu_time_us":8280,"apply.run_wall_time_us":180461,"num_ops":10,"prepare.queue_time_us":594932,"prepare.run_cpu_time_us":42090,"prepare.run_wall_time_us":323123,"raft.queue_time_us":282350,"raft.run_cpu_time_us":38163,"raft.run_wall_time_us":183909,"replication_time_us":1021045,"spinlock_wait_cycles":87554176,"thread_start_us":1255125,"threads_started":4,"wal-append.queue_time_us":249577}]]}
I20260709 13:04:22.897511 11361 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:22.897464 11635 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:22.944001  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.945201 11636 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:22.886761 11634 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.912688 11586 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:22.945915  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:22.957125  9120 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:22.949287 11563 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:22.893414 11568 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:22.977613 11522 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:22.982335  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:22.983099  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 194698us
I20260709 13:04:22.899250 11483 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:22.996026  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 144126us
I20260709 13:04:23.027526 11595 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.013319  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 187001us
I20260709 13:04:23.024032  9114 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:23.024416  9121 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:23.061321  9117 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.886330 11694 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:04:22.994347 11361 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:23.063776  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 282723us
I20260709 13:04:22.997224 11635 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:23.054247  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:22.996778  9860 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:23.029244 11586 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:23.112116  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 47028us
I20260709 13:04:23.102952  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:23.167193  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 64332.1us
W20260709 13:04:23.068183 11695 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:04:23.126916 10890 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.103364 11692 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.375s	user 0.001s	sys 0.001s
I20260709 13:04:23.152271  9115 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.063282 11634 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:23.180716 10890 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.136186 11706 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:23.168152  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:23.173275  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:23.151428 11494 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:23.138052 11517 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:23.117048 11522 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.183408  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.127777 11595 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:23.216212  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.176862 11588 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:23.219337  9117 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:23.172712 11609 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43778 (request call id 86) took 3881 ms. Trace:
W20260709 13:04:23.197255  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 133862us
I20260709 13:04:23.170529 11635 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:23.224220 10890 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:23.222409  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.225755  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.225016  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:23.230926  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 57779.6us
I20260709 13:04:23.231669  9826 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:23.231678  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48477us
I20260709 13:04:23.230316  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.214337 11667 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.237994  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.222110  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:23.244630  9190 raft_consensus.cc:606] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:23.244989  9190 raft_consensus.cc:614] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:23.221081 11588 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:23.239986  9194 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:23.240621  9193 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:23.241291  9196 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:23.215968 11542 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:23.246081  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.249650  9191 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:23.250677  9115 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:23.246601  9198 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:23.252493  9197 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:04:23.252705  9189 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:23.225423 11654 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:23.215263 11483 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.263490  9198 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:04:23.264909 11701 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
W20260709 13:04:23.266620 11697 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:04:23.267889 11703 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 13:04:23.268267 11683 consensus_meta.cc:287] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.004s
I20260709 13:04:23.216212 11706 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:23.268890 11667 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.270833 11706 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:23.271301 11667 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:23.184309 11568 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.272315 11568 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:23.273307 11712 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:23.243477 11630 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:23.249477 11588 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:23.281155  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:23.257539 11679 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.001s
I20260709 13:04:23.279975  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.282853  9115 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:23.222697 11609 rpcz_store.cc:276] 0709 13:04:19.245285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:19.245538 (+   253us) service_pool.cc:224] Handling call
0709 13:04:19.968747 (+723209us) ts_tablet_manager.cc:1151] Deleting tablet 98d913b72e5b43e8b8f20fb99d0838c4
0709 13:04:19.968845 (+    98us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:04:22.371610 (+2402765us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:22.668668 (+297058us) tablet_metadata.cc:665] Metadata flushed
0709 13:04:23.126484 (+457816us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":832607,"tablet-delete.queue_time_us":508641,"thread_start_us":284089,"threads_started":1}
I20260709 13:04:23.285050 11595 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:23.285488 11563 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:23.285403 11531 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.227536 11517 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.305929 11517 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:23.308123  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.311995  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:23.312223  9121 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:23.312873  9196 raft_consensus.cc:2410] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 13:04:23.268920 11696 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:04:23.294680  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.316391  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:23.317077  9189 raft_consensus.cc:2368] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5.
I20260709 13:04:23.315238 11634 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:23.287016 11498 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:23.296933  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:23.321058  9117 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:23.280458 11640 consensus_meta.cc:287] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.576s	user 0.021s	sys 0.000s
I20260709 13:04:23.360334 11640 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:23.361096 11702 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.468s	user 0.001s	sys 0.002s
I20260709 13:04:23.363106 11640 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:23.286793 11699 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
W20260709 13:04:23.364624 11682 log.cc:927] Time spent T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.899s	user 0.006s	sys 0.001s
W20260709 13:04:23.300750  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.366384  9858 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:04:23.366765  9828 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:23.367580  9194 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:23.303157  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:23.296053  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 44713.3us
I20260709 13:04:23.323675 11498 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.326185  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:23.377836  9197 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:23.378903  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.326941  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:23.379842 11714 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:23.332263  9117 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:23.334049  9121 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:23.381033 11714 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:23.381358 11483 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:23.327643  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:23.383697 11714 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:23.341251  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:23.341566  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:23.341923  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:23.345600  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:23.346378  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:23.386242  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:23.326833  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:23.348135 11226 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:23.349709 11522 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:23.378957  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:23.380131  9195 raft_consensus.cc:2376] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7. Re-sending same reply.
I20260709 13:04:23.294174 11679 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.356176  9005 catalog_manager.cc:5028] TS ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): tablet 98d913b72e5b43e8b8f20fb99d0838c4 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:04:23.386943  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:23.390007  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:23.326256 11707 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:04:23.384055 11595 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:23.391492  9859 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.400650 11595 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:23.335243  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.401582 11595 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.402175  9198 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:23.403081  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:23.391808  9854 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.404618 11654 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:23.405443 11654 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:23.405787 11654 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:23.406164 11654 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:23.391109  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:23.408656  9861 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:23.347404 11531 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:23.391471  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.394096  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:23.393455  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:23.394737  9191 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:23.410553  9117 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:23.395048 11226 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:23.395855  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:23.412612 11226 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.397820  9853 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:23.397593 11679 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:23.414492  9854 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
W20260709 13:04:23.390638  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49506.5us
I20260709 13:04:23.434139  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:23.398764  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.436491 11531 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:23.396595 11713 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:23.437418 11713 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:23.339138  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.397167  9189 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:23.437920 11713 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:23.391919 11333 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:04:23.438866  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.439147  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.439293 11333 raft_consensus.cc:697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:23.390076 11714 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:23.440140  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:23.440500  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:23.440565 11714 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.428822  9859 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:23.441237  9263 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:23.430881 11361 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.428714 11586 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.874835  9828 leader_election.cc:400] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5.
I20260709 13:04:23.877178  9828 leader_election.cc:403] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:04:23.876480 11595 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:23.878026  9828 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.568s (SENT) [suppressed 83 similar messages]
I20260709 13:04:23.878717  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:23.879788 11588 consensus_meta.cc:287] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.000s
W20260709 13:04:23.481782  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 248955us
I20260709 13:04:23.880388 11588 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.882866 11588 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:23.875165  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 441120us
I20260709 13:04:23.884101 11226 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:23.875504  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.887061  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.748648  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:23.888193  9860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.008s	sys 0.004s
I20260709 13:04:23.888244  9272 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:04:23.577579  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.889153  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:23.890237  9114 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:23.358894  9018 catalog_manager.cc:5697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 4, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:23.631495  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.442340  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:23.512780  9826 leader_election.cc:336] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.702s (SENT)
I20260709 13:04:23.892530  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:23.893270  9862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.008s	sys 0.006s
W20260709 13:04:23.893730  9862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.009s	sys 0.006s
W20260709 13:04:23.513278  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 126413us
I20260709 13:04:23.753629 11483 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:23.888652  9860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.008s	sys 0.004s
I20260709 13:04:23.903473  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 1995) took 1075 ms. Trace:
I20260709 13:04:23.905607  9263 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
W20260709 13:04:23.909531  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.765142 11333 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.529793 11568 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:23.525758 11494 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:23.942718  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:23.577195 11714 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:23.946188  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:23.520706 11640 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:23.941857  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:23.908627 11654 consensus_meta.cc:287] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.000s
W20260709 13:04:23.947979  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 435239us
I20260709 13:04:23.904016  9266 rpcz_store.cc:276] 0709 13:04:22.827893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:22.828023 (+   130us) service_pool.cc:224] Handling call
0709 13:04:22.846708 (+ 18685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:22.846946 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:23.148119 (+301173us) log.cc:844] Serialized 14 byte log entry
0709 13:04:23.903017 (+754898us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b1800192d60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f765628c4bc 00007f76535da48b 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287
0709 13:04:23.903102 (+    85us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:23.903144 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:23.903193 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:23.903267 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:23.903407 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":377694,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":117572,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":5555456,"thread_start_us":343176,"threads_started":2}
I20260709 13:04:23.520169 11679 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:23.959117  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:23.897222 11588 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5.
I20260709 13:04:23.966907  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:23.945245 11542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.012s	sys 0.007s
W20260709 13:04:23.974377 11542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.013s	sys 0.007s
I20260709 13:04:23.974685 11542 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:23.975682 11542 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.437847 11635 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.979782  9115 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:23.981567 11635 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:23.982403 11595 consensus_queue.cc:1059] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:23.983209  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:23.983868  9190 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:23.989112 11586 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:23.989840 11586 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:23.991195  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:23.992458 11361 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:23.994561 11721 raft_consensus.cc:3055] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:23.995029 11721 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:23.996008 11721 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:23.997354  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:23.968760  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:23.969435  9269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.004s	sys 0.019s
I20260709 13:04:23.971127  9195 raft_consensus.cc:2410] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:23.967869  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:24.001122  9269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.004s	sys 0.019s
I20260709 13:04:24.001638  9197 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:23.971668  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:23.945046 11642 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.012s	sys 0.005s
I20260709 13:04:24.009369  9190 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:24.002336  9114 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:24.002804  9826 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.046s (SENT)
W20260709 13:04:24.009577 11642 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.012s	sys 0.006s
I20260709 13:04:24.028002  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:24.028558  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.032195  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.036444  9194 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:24.040498  9192 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:24.047986  9198 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
W20260709 13:04:24.059466  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56688.2us
I20260709 13:04:23.911175 11483 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.016494  9855 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:04:24.041121  9190 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:24.068717  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.042057 11361 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:24.073858  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.071306  9117 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:24.002020 11654 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.009717 11679 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.000869 11640 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.077373  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:24.068540 11586 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.070180  9195 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:24.090400  9196 raft_consensus.cc:2410] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:24.077982  9197 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:24.080760  9189 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:04:24.079552  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 195477us
I20260709 13:04:24.098031  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.098905  9191 raft_consensus.cc:2410] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 13:04:24.100770  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 222753us
I20260709 13:04:24.102654  9114 raft_consensus.cc:2410] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:04:24.108635  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 227815us
W20260709 13:04:24.095263 11713 consensus_meta.cc:287] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.001s
I20260709 13:04:24.116549  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.077054 11361 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.121110 11713 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.127848  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.128911  9197 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:24.115041 11498 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:24.095026 11586 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:24.136730  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 262411us
I20260709 13:04:24.153431 11715 consensus_queue.cc:1059] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:24.154503 11712 raft_consensus.cc:3055] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:24.154860 11712 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:24.160145  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:24.166364  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 270628us
I20260709 13:04:24.167441  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:24.178706  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.181849  9189 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:24.180961  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:24.155613 11712 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.196069  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:24.200831  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:24.203138  9164 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:24.195065  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.205833  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:24.190656  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:24.211004  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:24.226209  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.227180  9189 raft_consensus.cc:2393] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:24.187566 11494 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:24.237003 11640 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:24.193933 11568 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:24.239662 11722 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:04:24.240797 11361 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:24.224177 11498 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:04:24.254607  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 58645.3us
I20260709 13:04:24.210569 11725 raft_consensus.cc:2764] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:04:24.205549 11517 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:04:24.259720  9263 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:24.286947  9189 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:04:24.190619 11729 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:04:24.244995 11542 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.283416  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 92748.3us
W20260709 13:04:24.274005  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 70956.4us
I20260709 13:04:24.261382 11531 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:24.301108  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.258456 11728 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:24.259393 11724 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:24.271183 11640 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:24.271765 11712 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:24.288987 11483 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:24.298396 11642 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:24.316246  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.323355  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:24.324332  9859 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:24.307358 11531 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.326704  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.327661  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.333324  9860 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:24.322474  9196 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:24.303097 11679 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:24.332129  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 131437us
I20260709 13:04:24.333770  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:24.337355  9854 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:24.309778 11728 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:24.282817 11494 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.344743  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 428) took 1015 ms. Trace:
I20260709 13:04:24.318049 11640 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.346410  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:24.350121  9859 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:24.353075  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.336195 11531 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:24.363731 11654 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:24.365959 11226 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:24.366443 11226 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:24.368527 11740 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:24.372933  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.373718  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:24.373977 11512 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:24.374326 11512 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:24.374953  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.375162 11512 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.376307 11713 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:24.376523 11512 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:24.340281 11517 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:04:24.349352 11724 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:04:24.345577 11728 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:24.379328 11494 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:24.362040 11586 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:24.307716 11542 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.381201 11721 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:24.354409 11568 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.353919  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.353822 11729 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:24.387924  9862 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:24.340220 11731 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:24.404141 11731 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:24.405792 11729 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:24.375837  9195 raft_consensus.cc:2410] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:24.381845 11542 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:24.427448  9089 connection.cc:256] RPC call timeout handler was delayed by -1.5863509960000215s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3186s for a response
W20260709 13:04:24.428274  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0183816089999596s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 13:04:24.433018  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 136411us
I20260709 13:04:24.433878  9859 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:24.436072  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:24.437808  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.438288  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.438628  9853 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:24.438895  9860 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:24.381235 11640 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:24.407449 11713 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:24.367452 11226 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.337415  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:24.459088  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:24.459832  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:24.463074  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.466645  9859 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:24.467015  9860 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:04:24.467308  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 236663us
W20260709 13:04:24.465318  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 127986us
W20260709 13:04:24.465265  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.468710  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:24.469105  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.469246  9195 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:24.470090  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.470225  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.345227  9193 rpcz_store.cc:276] 0709 13:04:23.325058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:23.395471 (+ 70413us) service_pool.cc:224] Handling call
0709 13:04:23.872108 (+476637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:23.872325 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:24.212611 (+340286us) log.cc:844] Serialized 14 byte log entry
0709 13:04:24.340304 (+127693us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:24.340356 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:24.340411 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:24.340476 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:24.340610 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18510,"prepare.queue_time_us":463904,"raft.queue_time_us":19103,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":3027,"spinlock_wait_cycles":172416,"thread_start_us":443635,"threads_started":2}
W20260709 13:04:24.480077  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 133799us
I20260709 13:04:24.480265  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:24.480722 11735 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:24.481087 11735 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:24.481624  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:24.481793  9862 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:24.481855 11735 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.482939 11735 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:24.483382  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.484213  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:24.484315  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:24.485180  9114 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:24.485386  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:24.485898  9117 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:04:24.486704 11714 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:24.483007 11531 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.489822 11531 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:24.490307  9121 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:24.491587  9826 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:24.492713 11654 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:24.494203  9117 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:24.494836 11635 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:24.495301 11635 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.433472 11729 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.452709 11731 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:24.480263  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:24.452383 11713 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:24.502863  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:24.503194  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.498713  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:24.503724  9117 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:24.504244  9115 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:24.499467  9123 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:24.436048 11542 consensus_queue.cc:1059] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:04:24.472138 11517 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:24.407644 11740 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.522459  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:24.525269 11730 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:04:24.526110  9123 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:24.525314  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.476389  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:24.531060  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:24.531703  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.532526  9860 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:24.532646  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.533809 11713 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.535157 11713 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:24.535933 11586 raft_consensus.cc:3055] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:24.536309 11586 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:24.537140 11586 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.538311 11586 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:24.541082  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.541805  9860 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:24.518391 11333 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:04:24.545212  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.545864  9862 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:24.547158  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.549212 11740 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:24.550948  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.551544  9862 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:24.556838  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:24.558794  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:24.559615  9123 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:24.464432 11741 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 13:04:24.567858  9238 connection.cc:256] RPC call timeout handler was delayed by -2.017200331999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2864s for a response
I20260709 13:04:24.506712 11640 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:24.569109  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.600533  9196 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 13:04:24.596231  9238 connection.cc:256] RPC call timeout handler was delayed by -1.6012528420000081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 13:04:24.602742  9238 connection.cc:256] RPC call timeout handler was delayed by -1.1413528420000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2642s for a response
I20260709 13:04:24.605106  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:24.606716  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.607820  9189 raft_consensus.cc:2410] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 13:04:24.624961  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 143743us
I20260709 13:04:24.627110  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:24.634382  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:24.550028 11517 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:24.635186  9197 raft_consensus.cc:2376] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3. Re-sending same reply.
I20260709 13:04:24.644254  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:24.647560  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 170807us
I20260709 13:04:24.595431 11498 raft_consensus.cc:697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:24.661392  9266 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:24.665866  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.672422  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.672937  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.673662  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:24.673023  9853 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:24.666221  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.673009  9195 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:24.674626  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:24.624732 11745 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:24.676308  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.691953  9193 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:24.604302 11640 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.837576  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:24.721935  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:24.904778  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:24.911932  9264 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
W20260709 13:04:24.670852 11752 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.208s	user 0.001s	sys 0.001s
I20260709 13:04:24.915972  9272 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:25.348811 11586 consensus_meta.cc:287] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.810s	user 0.012s	sys 0.000s
I20260709 13:04:24.791375  9114 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:24.899554 11517 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:24.791157  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:25.352514  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:24.766227  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:24.961038  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.353199  9861 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
W20260709 13:04:24.987479  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.978389  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:24.722770 11642 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:24.790913  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.776549  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:24.904315  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 402985us
W20260709 13:04:24.766831  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.354903  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.995018  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:24.723291  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.965018  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.765630  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:24.910893  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 127003us
I20260709 13:04:24.810202  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:24.910497 11745 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:25.356940  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:24.668242 11333 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.689s
I20260709 13:04:24.981544  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:24.827934 11754 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:25.357434  9089 leader_election.cc:336] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.503s (SENT)
I20260709 13:04:24.721592  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:24.964345  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:25.310639  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 379609us
W20260709 13:04:24.872287  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.136221  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.352517  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:25.085877  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:24.903498 11728 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.525s	user 0.005s	sys 0.000s
W20260709 13:04:24.991508 11735 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.003s
W20260709 13:04:25.109440 11712 consensus_meta.cc:287] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.790s	user 0.006s	sys 0.003s
I20260709 13:04:25.494782  9267 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:25.364475  9238 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.522s (SENT)
I20260709 13:04:25.512635  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:25.498406  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.499265  9270 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:25.499713  9860 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:04:25.513454  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.500478  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:25.514045  9195 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:24.982051 11512 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.501574  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 506683us
I20260709 13:04:25.501969  9193 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:25.363663  9272 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:25.495836  9194 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:25.059545 11679 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:25.157385 11713 consensus_meta.cc:287] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.000s
I20260709 13:04:25.526475  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:25.504977  9087 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.687s (SENT)
I20260709 13:04:25.504760  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.507014  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.500176  9198 raft_consensus.cc:2376] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5. Re-sending same reply.
I20260709 13:04:25.529258  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:25.525761  9269 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:25.508198  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:25.509238  9828 leader_election.cc:336] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.865s (SENT)
I20260709 13:04:25.532369  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1828) took 1103 ms. Trace:
I20260709 13:04:25.532766  9196 rpcz_store.cc:276] 0709 13:04:24.428604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.428767 (+   163us) service_pool.cc:224] Handling call
0709 13:04:25.532170 (+1103403us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400143554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:25.532312 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375513856}
I20260709 13:04:25.533977  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.535030  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.536432  9263 raft_consensus.cc:2410] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:25.537247  9197 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:25.538300  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.530836  9828 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:25.541213  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.542392  9196 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:25.521073  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.543610  9272 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:25.521415  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:25.527315  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 169919us
W20260709 13:04:25.544674  9089 leader_election.cc:336] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.581s (SENT)
I20260709 13:04:25.501036  9265 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:25.544929  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:25.508117  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.528406  9191 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:25.546437  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:25.539700  9271 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:25.550001  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:25.510234 11568 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.501099 11586 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:25.567256  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.568279  9191 raft_consensus.cc:2410] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:25.568991  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:25.554030  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.553818  9855 raft_consensus.cc:2410] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:25.584125 11634 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:25.584805 11634 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.524226 11542 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:25.593537  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.004022 11498 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.598443  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.599305  9191 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 3: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:25.548765 11512 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.600920 11640 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:25.601702 11640 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:25.601961 11512 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:25.602548 11640 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:25.602797  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.603035 11640 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.603726  9198 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:04:25.605813 11654 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.606436 11654 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.589794  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:25.616040  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.616412  9195 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:25.616753  9264 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:25.616830  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:25.617558  9270 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.555676  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:25.510582 11531 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.557636  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:25.560717  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:25.565696  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:25.564376  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 59417us
I20260709 13:04:25.557178  9189 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:25.624300  9193 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:25.625770  9197 raft_consensus.cc:2368] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5.
I20260709 13:04:25.587602 11634 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:25.634305  9189 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:25.505026 11754 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.633145  9265 raft_consensus.cc:2368] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5.
I20260709 13:04:25.632618  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.556062 11712 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.603547 11640 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.588584 11713 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:25.652590  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:25.635514  9264 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:25.510769 11728 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.657994  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.658710 11728 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:25.658869  9197 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:25.661441  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:25.632892 11542 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:25.663429 11542 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.666904 11642 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.539304 11679 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:25.676669  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48970us
I20260709 13:04:25.677899  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.679116  9269 raft_consensus.cc:2410] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:25.678011  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:25.680181  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:25.680595  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:25.681393  9116 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:25.682519  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:25.683363  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:25.655920 11754 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:25.653577  9265 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.637122  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:25.726166  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.719945  9119 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:25.718675  9826 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.174s (SENT)
I20260709 13:04:25.728621  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.729074  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:25.718643  9272 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.725455  9270 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:25.732908  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:25.733577  9270 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.744553  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:25.745595  9123 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:25.746868  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:25.747654  9116 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 2.
I20260709 13:04:25.612524 11586 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.752374  9164 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:25.753813  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 149012us
I20260709 13:04:25.754115  9115 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:25.755805  9121 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:25.632688 11531 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:25.752583  9826 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.236s (SENT)
I20260709 13:04:25.758482  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:25.759534  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:25.612690 11568 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:25.650801 11712 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.807595  9117 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:04:25.798658  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 46389.8us
I20260709 13:04:25.813090  9272 raft_consensus.cc:2410] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:04:25.813710  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 61183.2us
I20260709 13:04:25.813740  9118 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 13:04:25.815464  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 294099us
I20260709 13:04:25.816013  9264 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:25.816181  9825 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:25.814075  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:25.819705  9190 raft_consensus.cc:2410] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:25.822964  9272 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:04:25.823928  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 67974.4us
I20260709 13:04:25.825088  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:25.826176  9193 raft_consensus.cc:2410] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:25.789561 11745 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:25.717905 11706 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:25.819114 11754 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:25.802119  9270 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
W20260709 13:04:25.829941  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 320720us
W20260709 13:04:25.830276  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 465834us
W20260709 13:04:25.856770  9238 leader_election.cc:336] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.964s (SENT)
I20260709 13:04:25.857285  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:25.852522  9161 connection.cc:256] RPC call timeout handler was delayed by -1.9042732149999892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
W20260709 13:04:25.858315  9161 connection.cc:256] RPC call timeout handler was delayed by -1.8637732149999806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
W20260709 13:04:25.858906  9161 connection.cc:256] RPC call timeout handler was delayed by -1.7116732150000189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
W20260709 13:04:25.859405  9161 connection.cc:256] RPC call timeout handler was delayed by -1.5244732150000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
I20260709 13:04:25.834640  9272 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
W20260709 13:04:25.859988  9161 connection.cc:256] RPC call timeout handler was delayed by -1.4884732149999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:04:25.864593  9161 connection.cc:256] RPC call timeout handler was delayed by -1.4866732149999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
W20260709 13:04:25.865200  9161 connection.cc:256] RPC call timeout handler was delayed by -1.4839732149999918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 13:04:25.865689  9161 connection.cc:256] RPC call timeout handler was delayed by -1.4686732150000239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2054s for a response
W20260709 13:04:25.868046  9161 connection.cc:256] RPC call timeout handler was delayed by -1.4164732149999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 13:04:25.878446  9161 connection.cc:256] RPC call timeout handler was delayed by -1.3786732149999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2154s for a response
I20260709 13:04:25.866262  9266 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:25.869313  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:25.861850 11727 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.835700 11706 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:25.873430 11778 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:25.874990 11586 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:25.875291  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.876700  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.904031  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 144589us
I20260709 13:04:25.909754  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:25.870551  9272 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:04:25.880023  9087 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.143s (SENT)
W20260709 13:04:25.893716  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:25.911443  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.895249  9161 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:25.852653 11772 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.841255 11754 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.861342 11679 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:25.871009 11498 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:25.843001 11713 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:04:25.919859 11769 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:25.910580  9089 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:25.851362 11745 consensus_queue.cc:1059] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:04:25.919931 11586 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:25.924327 11747 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.460s	user 0.010s	sys 0.021s
I20260709 13:04:25.926398  9267 raft_consensus.cc:2410] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:25.944598  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 565) took 1195 ms. Trace:
I20260709 13:04:25.944999  9267 rpcz_store.cc:276] 0709 13:04:24.748973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.749237 (+   264us) service_pool.cc:224] Handling call
0709 13:04:25.944538 (+1195301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:25.930692 11777 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.931501 11783 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:25.931895  9161 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:25.931985 11775 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:25.933104 11634 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:25.937165  9265 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:25.935952 11498 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.941568  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:25.942373  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.921236 11754 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:25.946882 11789 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:25.946820  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 130712us
I20260709 13:04:25.989915  9825 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:25.990752  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:25.904606 11727 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.938341 11679 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.960337 11783 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:25.953866  9862 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 1. Current term is 2. Ops: []
I20260709 13:04:25.954289 11793 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:25.954537  9828 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:25.939103 11778 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:25.959982  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 103259us
W20260709 13:04:25.962280  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.000334  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.000463  9861 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:25.942207 11586 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:25.982797  9161 leader_election.cc:400] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4.
I20260709 13:04:25.932111 11773 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:04:26.008677 11586 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:25.998701 11783 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.009334  9861 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:04:25.937829 11640 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.992914  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.010583  9859 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:25.921036 11772 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:25.960456 11568 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:25.950068  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.980425 11728 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:25.940502 11769 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:25.962011 11791 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:25.988025 11498 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:25.981315 11706 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.059314 11706 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:25.989032 11642 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:26.002396  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:26.061564  9860 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:26.021311  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:26.022692  9859 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:26.063957  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:25.998803 11788 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:26.016733 11728 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:25.999956 11531 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.028211 11773 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:25.987682 11517 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 13:04:26.029629  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 75231.3us
I20260709 13:04:26.127735  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:26.030306  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 161081us
I20260709 13:04:26.032645 11226 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:26.012312 11778 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:26.038586  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:25.995427 11779 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:04:26.041327  9857 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:04:26.042222  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.136894  9854 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:26.044469  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:26.137842  9853 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:04:25.998265 11785 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:26.003352 11793 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.011019 11789 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.045553  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.153463  9858 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:26.046100 11790 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.046561  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.016614 11634 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:04:26.048839 11635 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:26.053083 11770 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:04:25.998368 11679 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.173738 11769 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:26.174315 11679 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:26.174408 11769 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:25.959995 11777 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.175230  9192 raft_consensus.cc:2410] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:26.176637 11777 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:26.011200 11712 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.058058 11791 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:26.177320 11791 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:26.178351  9853 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:26.178272 11791 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:26.179435 11791 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.058404 11772 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:26.179818 11333 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.180372 11333 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.126425 11773 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:26.182116 11773 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.183364 11773 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:25.943473 11784 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:26.188000 11784 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.188480 11784 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:26.189342 11793 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:26.191357 11762 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.855s	user 0.002s	sys 0.000s
I20260709 13:04:26.013561 11727 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.069988  9164 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:26.192406  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 122493us
I20260709 13:04:26.193522 11483 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:26.193915 11483 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.194288 11483 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:26.194355  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:26.132716  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 132760us
W20260709 13:04:26.145730  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 160963us
I20260709 13:04:26.068109 11788 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:26.146767  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 125530us
W20260709 13:04:26.142815  9826 leader_election.cc:336] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.823s (SENT)
I20260709 13:04:26.035331 11783 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.072818 11728 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:26.213272  9267 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:26.148737 11785 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.158535 11789 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.158583 11790 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.160310 11774 raft_consensus.cc:3055] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:26.127862 11642 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:26.227375 11774 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:26.228293 11642 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.228490 11774 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:26.229761 11774 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.199733  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.232115  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:26.232409  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 89649us
W20260709 13:04:26.233207  9826 leader_election.cc:336] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.974s (SENT)
I20260709 13:04:26.197237  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:26.203680  9264 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:26.233608 11498 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.127450 11776 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:26.235054 11776 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:26.236261  9267 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:26.236699  9267 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:26.204983  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:26.237094 11793 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.237527  9265 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
W20260709 13:04:26.205155  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 51767.8us
I20260709 13:04:26.238758  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:26.210773  9860 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:26.157058  9856 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 3: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:26.216390 11634 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:26.221132  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.221724  9272 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:26.223969  9269 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:26.240530 11778 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:26.278945  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:26.303843  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:26.305204  9266 raft_consensus.cc:2410] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:04:26.306075  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:26.304265 11640 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:26.288489  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 86809us
W20260709 13:04:26.241794  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 44649.4us
I20260709 13:04:26.233608  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:26.323132  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 89952.2us
I20260709 13:04:26.242906  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.307457 11634 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.319423  9266 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:26.222216 11777 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.202841 11788 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:26.224671 11785 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:26.332283  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 204641us
I20260709 13:04:26.344974  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.332680  9161 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:26.277801 11791 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:26.306713 11679 raft_consensus.cc:2764] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:04:26.277107 11568 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:26.329957 11634 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:26.383813  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:26.336938  9826 leader_election.cc:336] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.967s (SENT)
W20260709 13:04:26.385510  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48582.4us
I20260709 13:04:26.240669 11498 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.340281 11790 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.326251 11640 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:26.340529 11586 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.213828 11635 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 13:04:26.213292 11772 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 13:04:26.214277 11226 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.116s
W20260709 13:04:26.341892  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 99096.9us
I20260709 13:04:26.404503  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.405094  9117 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:26.346577  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:26.351400 11728 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:26.376070  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.413601  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:26.414186  9853 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:26.381448  9266 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:26.394094  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.382085 11568 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:26.334021 11779 consensus_queue.cc:1059] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:04:26.399720 11586 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:26.352239 11517 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:04:26.402354  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:26.428565  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.428798  9118 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.330346 11796 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.240655 11784 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.431769 11779 raft_consensus.cc:3055] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:26.432334 11779 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:26.432494 11813 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:26.432984 11813 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:26.433653 11784 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.433305 11779 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.433859 11813 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:26.434669 11779 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:26.406462  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.412040  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.417725  9114 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:26.421026  9089 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:26.403940  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.393067 11791 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:26.461535 11791 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.462127  9267 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.463128  9267 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:26.383328 11642 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.464066 11642 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.411165 11804 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:26.465425 11794 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:26.465721 11804 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:26.465842 11794 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:26.466707 11804 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:26.344161 11706 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
W20260709 13:04:26.467724  9263 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 561) took 1841 ms (client timeout 2174 ms). Trace:
W20260709 13:04:26.468197  9263 rpcz_store.cc:269] 0709 13:04:24.626698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.626810 (+   112us) service_pool.cc:224] Handling call
0709 13:04:26.467519 (+1840709us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b54003fcfd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:26.467671 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2089052544}
I20260709 13:04:26.400326  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:26.389259 11795 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:26.470350  9270 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:26.352010 11777 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.472029  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1646) took 1117 ms. Trace:
I20260709 13:04:26.472739  9860 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:04:26.423676 11568 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:26.429262  9117 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:26.352144 11776 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:26.440469  9856 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:26.431684 11783 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.404874  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:04:26.477826  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 73053.4us
I20260709 13:04:26.441656 11789 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.445340  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 435) took 1472 ms. Trace:
I20260709 13:04:26.480665  9192 rpcz_store.cc:276] 0709 13:04:24.972853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.972995 (+   142us) service_pool.cc:224] Handling call
0709 13:04:26.445151 (+1472156us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54001d5954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:26.445278 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1564720384}
I20260709 13:04:26.445722 11728 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:26.446108  9271 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 562) took 1819 ms (client timeout 2236 ms). Trace:
W20260709 13:04:26.452239 11764 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.732s	user 0.004s	sys 0.003s
I20260709 13:04:26.424667 11772 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.452721 11806 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:26.424499 11498 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 13:04:26.458555  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 360016us
I20260709 13:04:26.513164  9268 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:04:26.515084  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.382443 11793 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:04:26.466703 11794 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.518534  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:26.518618 11794 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:26.472344  9270 rpcz_store.cc:276] 0709 13:04:25.354744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.853384 (+498640us) service_pool.cc:224] Handling call
0709 13:04:26.470007 (+616623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:26.470774 (+   767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.471855 (+  1081us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b54003fcfd0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:26.471976 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1417244928}
I20260709 13:04:26.520064  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.521416  9115 pending_rounds.cc:85] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 2
I20260709 13:04:26.521765  9115 pending_rounds.cc:107] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:04:26.522014  9115 raft_consensus.cc:2889] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:26.501161  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:26.526871 11333 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:26.496901  9271 rpcz_store.cc:269] 0709 13:04:24.627008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.627117 (+   109us) service_pool.cc:224] Handling call
0709 13:04:26.445965 (+1818848us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b54000fa5d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:26.446059 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1990073088}
I20260709 13:04:26.473361 11777 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:26.503978  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:26.505167  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:26.505694 11754 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
W20260709 13:04:26.509318  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 315028us
W20260709 13:04:26.521299  9855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 825) took 2086 ms (client timeout 1704 ms). Trace:
I20260709 13:04:26.467403 11642 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.532058  9267 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:26.540359  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.545109  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:26.558040  9271 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:26.546940  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.559182  9123 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:26.507184 11796 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:26.541769  9264 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:04:26.555351  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 208846us
I20260709 13:04:26.562987  9123 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:26.563397  9161 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:26.565649  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.569077  9117 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
W20260709 13:04:26.569540  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.528689 11789 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:26.565083  9235 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:26.510861 11806 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:26.560439  9164 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:26.581454  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.581599  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.582401  9164 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:26.572554  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 151641us
I20260709 13:04:26.583135  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.583388  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:26.573417  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.583913  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:26.588745  9089 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:26.589049  9122 raft_consensus.cc:2393] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:26.505726 11736 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:04:26.510932 11706 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
W20260709 13:04:26.591858  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.556478 11775 raft_consensus.cc:3055] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:04:26.603417  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.601666  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 228532us
I20260709 13:04:26.595283  9264 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:04:26.604084  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.604442  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.556231 11333 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:26.562381 11796 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.606259  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.590387 11642 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:04:26.583215 11789 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:26.619463  9235 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:26.620833  9190 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 440) took 1646 ms (client timeout 1568 ms). Trace:
W20260709 13:04:26.626399  9190 rpcz_store.cc:269] 0709 13:04:24.974343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:25.505431 (+531088us) service_pool.cc:224] Handling call
0709 13:04:26.620563 (+1115132us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54004173d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:26.620743 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663756544}
I20260709 13:04:26.621639  9197 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:26.585639 11783 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:26.590937 11498 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:26.603618 11775 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:26.592873 11798 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:26.559294 11754 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.631810  9089 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:26.568367 11728 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:04:26.616263 11774 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:26.629145  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.609035 11517 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:04:26.555855  9855 rpcz_store.cc:269] 0709 13:04:24.434983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:24.436677 (+  1694us) service_pool.cc:224] Handling call
0709 13:04:26.521095 (+2084418us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54005a9454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:26.521227 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1868267392}
I20260709 13:04:26.585469 11226 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:04:26.651636  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:26.642432  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 59365us
I20260709 13:04:26.617442 11796 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:26.648200 11706 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.633111 11712 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:26.600497 11799 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:04:26.653817  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:26.658399  9123 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:26.608407 11772 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:04:26.693392  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:26.630066 11635 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:04:26.667649  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.631094 11789 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:26.708827 11483 consensus_meta.cc:287] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.514s	user 0.005s	sys 0.006s
W20260709 13:04:26.683974  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.644965 11774 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:26.657258 11226 consensus_queue.cc:1243] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:26.663105 11517 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:26.640290 11798 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.698766  9828 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.699190  9118 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:26.735616 11586 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:26.736173 11586 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:04:26.736774  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 154439us
I20260709 13:04:26.737149 11808 raft_consensus.cc:3055] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:26.737705  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 70143.2us
I20260709 13:04:26.738122  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:26.738359  9856 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:26.738631  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:26.738937  9161 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.739238  9854 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.739559  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.739565  9857 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:26.740495  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:26.737635 11808 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:26.738940  9860 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:26.741894 11808 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:26.742631 11812 log.cc:927] Time spent T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:04:26.737123 11586 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.743480 11808 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.672269 11784 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.746917  9860 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:26.742540  9161 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:26.739266  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.696704  9089 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:26.732772  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 151918us
I20260709 13:04:26.737650  9164 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:26.703153 11777 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:26.719971 11793 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:26.748773  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.784315  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.698714 11712 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:26.727198 11774 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:26.774153  9854 raft_consensus.cc:2410] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:04:26.781476  9018 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36626 (request call id 358) took 15725 ms (client timeout 15000 ms). Trace:
I20260709 13:04:26.709236 11483 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.782047  9861 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 3: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:26.770256 11778 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.785293  9858 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 2: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:04:26.785218  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 88634.3us
I20260709 13:04:26.771140 11816 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:26.746070 11798 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:26.754177 11776 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:26.812726  9861 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:04:26.766301 11773 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:04:26.814661  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.784276 11793 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.800297 11800 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:04:26.815208  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.816011  9089 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:26.832515  9828 connection.cc:256] RPC call timeout handler was delayed by -1.0220888480000099s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 13:04:26.835052  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 92574.5us
I20260709 13:04:26.793843 11772 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:26.811904 11654 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:26.820644  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.849503  9861 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:26.850535  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:26.800411 11712 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:26.707052 11775 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:26.747452 11785 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.830895  9854 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:26.840888  9858 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:26.812330  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:26.855774  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.813298 11817 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.854620  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.856783  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.813858 11777 raft_consensus.cc:697] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:26.838130 11757 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:04:26.869556  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.838958 11776 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:26.813596 11815 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:26.854010  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:26.840965 11793 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:26.856228 11226 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:26.843266 11635 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.847002 11728 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:26.822829 11778 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.876029  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.875373  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:26.878886 11775 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:26.877492  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.843132 11769 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.880092  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.918366  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:26.881490 11654 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:26.872530  9855 raft_consensus.cc:2376] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8. Re-sending same reply.
I20260709 13:04:26.883800 11483 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.884119  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.928843  9191 raft_consensus.cc:2368] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6.
I20260709 13:04:26.884760  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.861760 11819 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:26.854156 11517 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:26.890175  9116 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
I20260709 13:04:26.940385  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:26.941251  9191 raft_consensus.cc:2368] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8.
I20260709 13:04:26.838392 11818 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.883842  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:26.901163  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 163590us
I20260709 13:04:26.792471 11824 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:04:26.902698  9859 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:26.905639  9118 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:26.951438  9196 raft_consensus.cc:2410] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:26.854235 11805 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:26.904704  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:26.953905  9123 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:26.875306  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 59385.3us
I20260709 13:04:26.955694  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:26.860245 11800 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:26.917034  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 134717us
I20260709 13:04:26.875663 11757 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.920545  9193 raft_consensus.cc:2410] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:04:26.876194  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.891970 11776 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:26.909406 11333 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:26.907351 11817 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:04:26.885505  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.903720 11531 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:26.949021 11774 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:26.909904 11754 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:26.952255  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:26.952816  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:26.958985  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:27.109488  9235 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:26.960130  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.958367  9019 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 4 to 5, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 5 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:26.967053 11757 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:26.963667  9197 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:26.965298  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:27.114095  9118 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 4: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:26.923885  9122 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:26.971158  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:27.115068  9858 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 4: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:04:26.975203 11820 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:04:26.819784 11826 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.113s
I20260709 13:04:26.979027  9123 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:26.974886 11830 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:04:26.973184  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:27.009087  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:26.971360  9223 heartbeater.cc:646] Failed to heartbeat to 127.7.213.126:45141 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.7.213.126:45141 timed out after 15.000s (SENT)
W20260709 13:04:26.979995 11829 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
W20260709 13:04:26.975641 11821 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
W20260709 13:04:26.987689  9828 leader_election.cc:336] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.568s (SENT)
I20260709 13:04:26.953532 11805 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:26.900921 11827 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:04:26.969596 11825 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.968796  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:27.147152  9861 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:04:26.976536 11823 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:04:26.980789 11635 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:26.937624 11728 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:26.987612  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:27.171597  9122 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:27.172039  9857 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:27.086115  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 130489us
I20260709 13:04:26.929338 11801 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.337s
W20260709 13:04:26.813571  9018 rpcz_store.cc:269] 0709 13:04:11.056034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:11.056254 (+   220us) service_pool.cc:224] Handling call
0709 13:04:26.781400 (+15725146us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:24.243964 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:24.294506 (+ 50542us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:24.298559 (+  4053us) write_op.cc:435] Acquired schema lock
0709 13:04:24.304349 (+  5790us) tablet.cc:662] Decoding operations
0709 13:04:24.343447 (+ 39098us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:24.345080 (+  1633us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:24.349558 (+  4478us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:24.376588 (+ 27030us) tablet.cc:701] Row locks acquired
0709 13:04:24.376607 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:04:24.376760 (+   153us) write_op.cc:270] Start()
0709 13:04:24.376874 (+   114us) write_op.cc:276] Timestamp: P: 1783602264376710 usec, L: 0
0709 13:04:24.376894 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:04:24.377573 (+   679us) log.cc:844] Serialized 533 byte log entry
0709 13:04:25.324319 (+946746us) op_driver.cc:464] REPLICATION: finished
0709 13:04:25.664052 (+339733us) write_op.cc:301] APPLY: starting
0709 13:04:25.664164 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:25.664298 (+   134us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:25.664462 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:25.664742 (+   280us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:25.665011 (+   269us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:25.665089 (+    78us) write_op.cc:312] APPLY: finished
0709 13:04:25.665328 (+   239us) log.cc:844] Serialized 23 byte log entry
0709 13:04:25.665547 (+   219us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.665659 (+   112us) write_op.cc:454] Released schema lock
0709 13:04:25.665711 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:25.666181 (+   470us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:25.666214 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":320332,"apply.run_cpu_time_us":2110,"apply.run_wall_time_us":2111,"num_ops":1,"prepare.queue_time_us":232184,"prepare.run_cpu_time_us":12612,"prepare.run_wall_time_us":277360,"raft.queue_time_us":601430,"raft.run_cpu_time_us":183366,"raft.run_wall_time_us":312347,"replication_time_us":898247,"thread_start_us":654461,"threads_started":4,"wal-append.queue_time_us":271936,"wal-append.run_cpu_time_us":11215,"wal-append.run_wall_time_us":2007781}]]}
I20260709 13:04:26.962606 11483 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:26.962464  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.985567 11776 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:27.193306 11840 raft_consensus.cc:2764] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:04:27.194084 11840 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:26.935236 11769 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.112294  9191 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:27.111502  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.938403 11654 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.126670  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:27.104027 11531 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:27.136025  9192 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:27.142652  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 155002us
W20260709 13:04:27.109817 11634 consensus_meta.cc:287] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.714s	user 0.008s	sys 0.003s
I20260709 13:04:27.144622  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:26.982487 11727 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:27.127553 11790 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:04:27.157377  9019 catalog_manager.cc:5697] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 4 to 5, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 5 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:27.227008  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 100439us
I20260709 13:04:27.158156  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:27.227821  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:27.158589  9116 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:27.229921  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.230525  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.230651  9265 raft_consensus.cc:2376] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2. Re-sending same reply.
I20260709 13:04:27.231124  9264 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:27.107014 11777 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.159633 11806 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:27.161840 11568 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.000s
W20260709 13:04:27.163383 11828 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 13:04:27.140473 11799 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 13:04:27.258129 11831 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.400s	user 0.001s	sys 0.001s
I20260709 13:04:27.248142 11531 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.259720 11531 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:27.263252  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.263731  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.168289 11834 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:27.168649  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:27.178368  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:27.265630  9193 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:27.182293 11333 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.183880 11785 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:27.184777  9858 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:04:27.247352 11794 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.728s	user 0.013s	sys 0.000s
I20260709 13:04:27.190390 11805 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:27.200654  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.207727  9857 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:27.110127 11773 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.183157 11728 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.224723  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:27.232818  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.233237  9191 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:27.284284  9264 raft_consensus.cc:2410] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:27.234104  9197 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:27.174361 11754 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:04:27.236845  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.224419 11586 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:04:27.240681  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:27.242870  9826 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.581s (SENT)
I20260709 13:04:27.288452  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:27.288882  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.195115 11796 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.221138 11483 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:27.292811  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:27.293599  9197 raft_consensus.cc:2410] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:27.293718 11483 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:27.285418 11835 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.265154 11834 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:27.272027 11808 consensus_meta.cc:287] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.002s
I20260709 13:04:27.248104 11818 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.254618 11654 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:27.255113  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:27.234625 11725 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:27.248600  9019 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 3 to 5, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:27.195824 11800 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:27.256440  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.256861 11568 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:27.257306 11837 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:04:27.263361  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.224637 11727 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:27.186280  9089 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:27.273427  9087 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.656s (SENT)
W20260709 13:04:27.280375  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.279553  9861 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:27.238786  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.248978 11813 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.284452  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:27.286952  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.265192 11801 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:27.288818  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:27.289273  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 46440.3us
I20260709 13:04:27.193518 11798 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.290925  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.226732 11634 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.291925  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:27.294181 11785 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:04:27.250723  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 396792us
I20260709 13:04:27.252665  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.296164 11835 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.300606 11769 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:27.253589  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:27.280982 11806 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:27.305382  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 195970us
W20260709 13:04:27.516752  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.308804 11796 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:27.706660  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.706897  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:27.714265  9197 raft_consensus.cc:2410] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:04:27.714362  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.629838  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.715646  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.716636  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.717298  9264 raft_consensus.cc:2376] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4. Re-sending same reply.
W20260709 13:04:27.717805  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 444419us
I20260709 13:04:27.718645  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:27.719252  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.719874  9265 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:27.713398  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:27.721048  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.722384  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:27.715651  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.732802  9857 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:27.373461  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.291344 11586 consensus_queue.cc:1243] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer 94504d29c6744f38bb06943479df9466 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:04:27.776451  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 590209us
I20260709 13:04:27.779891  9191 raft_consensus.cc:2376] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5. Re-sending same reply.
I20260709 13:04:27.769994  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.780748  9269 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
W20260709 13:04:27.320570  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.780943  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:27.781848  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:27.704422 11847 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.284s	user 0.000s	sys 0.000s
I20260709 13:04:27.771138  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.715153  9268 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
W20260709 13:04:27.713554 11816 consensus_meta.cc:287] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.865s	user 0.002s	sys 0.008s
W20260709 13:04:27.772312  9826 leader_election.cc:336] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.668s (SENT)
I20260709 13:04:27.784302  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.794133  9266 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:27.794972  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.795454  9266 raft_consensus.cc:2376] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4. Re-sending same reply.
I20260709 13:04:27.796195  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.796720  9266 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:27.797513  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.798454  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.799147  9266 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:04:27.785033  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.799957  9270 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:27.801119 11843 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:04:27.730773  9828 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:27.810882  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:27.787519  9191 raft_consensus.cc:2410] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:27.317385  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:27.328817 11825 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:04:27.408767  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 129493us
I20260709 13:04:27.719365  9264 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
W20260709 13:04:27.822065  9238 leader_election.cc:336] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.572s (SENT)
I20260709 13:04:27.822402  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.822453  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:27.618167 11679 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:27.823509  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:27.789922  9265 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:27.824556  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:27.824576  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 93879us
I20260709 13:04:27.825378  9265 raft_consensus.cc:2410] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:27.826560  9265 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:27.789116  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.790479  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:27.651577 11841 log.cc:927] Time spent T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 13:04:27.644666 11712 consensus_meta.cc:287] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.771s	user 0.006s	sys 0.002s
W20260709 13:04:27.701188 11788 consensus_meta.cc:287] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.316s	user 0.011s	sys 0.000s
W20260709 13:04:27.819828 11531 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.000s
I20260709 13:04:27.812803 11816 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:27.729832 11789 consensus_meta.cc:287] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.021s	user 0.011s	sys 0.000s
W20260709 13:04:27.415805 11706 consensus_meta.cc:287] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.000s
I20260709 13:04:27.812994 11778 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:27.783723 11725 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:04:27.806967  9270 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:27.321655 11791 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:27.807623  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.866088  9266 raft_consensus.cc:2376] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4. Re-sending same reply.
I20260709 13:04:27.808200  9114 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:27.706084 11805 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:27.317027  9019 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 4 to 5, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 5 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:27.775846  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 50370.4us
I20260709 13:04:27.774617  9164 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:27.815137  9856 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:27.383785 11818 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.820686  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:27.879815  9268 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:27.287709 11773 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.489630 11774 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:27.833686  9190 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:27.834192  9269 raft_consensus.cc:2368] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5.
I20260709 13:04:27.788722 11635 raft_consensus.cc:2764] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:04:27.547469 11840 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:27.839344  9121 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:27.840332  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 49910.9us
I20260709 13:04:27.841842  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:27.791793  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:27.673586 11776 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:27.843428  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:27.844991  9238 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.577s (SENT)
I20260709 13:04:27.839608 11679 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.628239 11757 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.846849  9120 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:04:27.847819  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 101472us
I20260709 13:04:27.775522 11785 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:04:27.818085 11799 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:27.806409  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.939869  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:27.818317 11836 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.853423  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 799) took 1089 ms. Trace:
I20260709 13:04:27.847352 11712 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:27.853551 11706 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:27.853910 11789 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.849311 11788 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.855204 11790 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.835680 11808 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:27.859817  9115 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:27.861138 11727 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.833120 11777 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:04:27.842927 11783 consensus_meta.cc:287] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.209s	user 0.011s	sys 0.001s
I20260709 13:04:27.835523 11833 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.832499 11825 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:27.865757 11791 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:27.874423  9122 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:04:27.640108 11834 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:27.776980 11813 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:27.876667  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 102135us
I20260709 13:04:27.818548  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:27.884198  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:27.886008  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2013) took 1339 ms. Trace:
I20260709 13:04:28.014711  9114 rpcz_store.cc:276] 0709 13:04:26.552392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.562093 (+  9701us) service_pool.cc:224] Handling call
0709 13:04:26.567998 (+  5905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.866964 (+1298966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.885675 (+ 18711us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400254254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:27.885925 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2851910528}
I20260709 13:04:27.890313  9271 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:27.866238 11531 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.847433 11801 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:27.896662  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1901) took 1008 ms. Trace:
I20260709 13:04:28.019799  9119 rpcz_store.cc:276] 0709 13:04:26.888673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.903892 (+ 15219us) service_pool.cc:224] Handling call
0709 13:04:26.924696 (+ 20804us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.954450 (+ 29754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.954479 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.962265 (+  7786us) log.cc:844] Serialized 26 byte log entry
0709 13:04:27.228763 (+266498us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:27.228815 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.444826 (+216011us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b5400255654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:27.445911 (+  1085us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.887021 (+441110us) raft_consensus.cc:1685] finished
0709 13:04:27.887050 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.896597 (+  9547us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67881856,"thread_start_us":259474,"threads_started":1,"wal-append.queue_time_us":333401}
I20260709 13:04:27.865756 11827 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.897962  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1870) took 1022 ms. Trace:
I20260709 13:04:27.898962  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:28.021906  9190 rpcz_store.cc:276] 0709 13:04:26.875880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.876683 (+   803us) service_pool.cc:224] Handling call
0709 13:04:27.897728 (+1021045us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b5400068bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:27.897900 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075666176}
I20260709 13:04:27.898985  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 798) took 1150 ms. Trace:
I20260709 13:04:27.904822  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:28.022881  9856 rpcz_store.cc:276] 0709 13:04:26.748441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.748560 (+   119us) service_pool.cc:224] Handling call
0709 13:04:26.777533 (+ 28973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:27.876635 (+1099102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:27.898757 (+ 22122us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54007102d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:27.898926 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2385960832}
W20260709 13:04:28.023365  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 178406us
W20260709 13:04:28.024082  9238 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.609s (SENT)
I20260709 13:04:27.791958  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:28.024487  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:27.899712  9197 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:27.908456  9267 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:27.940999  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
W20260709 13:04:27.906281 11517 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.019s	sys 0.056s
I20260709 13:04:27.941718  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:28.028499  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 86844.7us
I20260709 13:04:27.943588  9860 rpcz_store.cc:276] 0709 13:04:26.766085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.767438 (+  1353us) service_pool.cc:224] Handling call
0709 13:04:26.767887 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:26.768547 (+   660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:26.768601 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:26.802585 (+ 33984us) log.cc:844] Serialized 26 byte log entry
0709 13:04:26.916826 (+114241us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:26.916871 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:26.916955 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:27.853215 (+936260us) raft_consensus.cc:1685] finished
0709 13:04:27.853238 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:27.853361 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":66245,"threads_started":1,"wal-append.queue_time_us":154395}
I20260709 13:04:27.944814  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:27.898458 11840 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:27.919546 11843 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:27.950006  9019 catalog_manager.cc:5697] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 2 to 4, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 4 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:27.957986  9194 raft_consensus.cc:2368] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9.
I20260709 13:04:27.921494 11568 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:27.987538  9268 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:27.998389  9272 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:27.947012 11839 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:27.974960 11586 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:27.994055 11783 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:28.024996  9263 raft_consensus.cc:2410] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:04:28.024468  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 73701.6us
I20260709 13:04:27.944288 11799 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:28.027091  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 128214us
I20260709 13:04:28.069607  9198 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:28.029361 11779 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.110409  9117 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
I20260709 13:04:28.019311 11727 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:28.060701  9192 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:28.103227  9019 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 2, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 2 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:28.061117 11852 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:04:28.118287  9192 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:28.061273 11857 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:28.063687 11333 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:27.948947 11836 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:27.990082 11790 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:28.028076 11517 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.020s	sys 0.057s
W20260709 13:04:28.065260  9238 leader_election.cc:336] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.822s (SENT)
I20260709 13:04:28.028023 11808 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:28.108326  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:28.032845  9196 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:04:28.056551 11798 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:28.031629 11634 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:28.058789 11840 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:28.116036 11586 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:28.116497 11801 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:28.153458  9089 leader_election.cc:400] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5.
I20260709 13:04:28.134337  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:28.002753 11833 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:28.094344 11846 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:28.136169  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.162352  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:28.137079  9264 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:28.025405 11531 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:28.030404 11827 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:28.116308 11862 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:04:28.175253  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 110018us
W20260709 13:04:28.175966  9238 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.965s (SENT)
I20260709 13:04:28.176334  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:28.115917 11839 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.066082 11834 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:28.150189  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.154743  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 138104us
W20260709 13:04:28.156585  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 280840us
W20260709 13:04:28.184126  9087 leader_election.cc:336] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.633s (SENT)
I20260709 13:04:28.184536  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:28.185055 11839 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:28.165903  9268 pending_rounds.cc:85] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 4
I20260709 13:04:28.189829  9268 pending_rounds.cc:107] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:04:28.190148  9268 raft_consensus.cc:2889] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:28.167661  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:28.174540  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:28.191175  9271 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:28.135535 11808 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:28.174064 11568 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:28.155676  9190 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:04:28.174824 11775 consensus_meta.cc:287] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.245s	user 0.008s	sys 0.008s
W20260709 13:04:28.217625  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.152092 11783 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:28.188472  9269 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:28.205219  9271 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:04:28.219503  9238 leader_election.cc:336] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.099s (SENT)
I20260709 13:04:28.195689  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:28.120698 11779 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:28.180109  9019 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 6 to 7, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), no longer has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 7 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:28.220413  9238 leader_election.cc:336] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Timed out: RequestConsensusVote RPC to 127.7.213.68:43523 timed out after 2.140s (SENT)
W20260709 13:04:28.225257  9238 leader_election.cc:336] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.172s (SENT)
W20260709 13:04:28.225785  9238 leader_election.cc:336] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Timed out: RequestConsensusVote RPC to 127.7.213.68:43523 timed out after 2.172s (SENT)
I20260709 13:04:28.181082 11725 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:28.174050 11843 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:28.195544 11793 consensus_meta.cc:287] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.282s	user 0.012s	sys 0.000s
W20260709 13:04:28.220570  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 185437us
I20260709 13:04:28.227865  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:28.230373  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.226176  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:28.212267 11808 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:28.213267 11568 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.120505 11791 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:28.221598 11840 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:28.239332  9269 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:28.220921 11776 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:28.217638 11801 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:28.174887 11800 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
W20260709 13:04:28.288048  9238 leader_election.cc:336] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.187s (SENT)
W20260709 13:04:28.255429  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.375509  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 135212us
I20260709 13:04:28.232311 11835 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:28.245314 11783 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:28.401823  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.372756  9826 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.546s (SENT)
I20260709 13:04:28.404268  9197 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:04:28.245077 11568 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:28.239459 11757 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:28.403456  9238 leader_election.cc:336] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Timed out: RequestConsensusVote RPC to 127.7.213.68:43523 timed out after 2.187s (SENT)
W20260709 13:04:28.254565 11815 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.334s	user 0.009s	sys 0.003s
I20260709 13:04:28.430954 11531 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:28.390622  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.244990 11679 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:28.440848  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:28.432510  9270 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:28.432837  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 60138.3us
I20260709 13:04:28.453457  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:28.243393 11793 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:28.243027 11859 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.269s
I20260709 13:04:28.455166  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:28.507058  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:28.818064  9114 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:04:28.812140  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 457) took 1079 ms. Trace:
W20260709 13:04:28.820556  9194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1867) took 1964 ms (client timeout 1822 ms). Trace:
W20260709 13:04:28.824304  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.823771  9194 rpcz_store.cc:269] 0709 13:04:26.856072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.856275 (+   203us) service_pool.cc:224] Handling call
0709 13:04:28.820275 (+1964000us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54001d5954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:28.820459 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2363060992}
I20260709 13:04:28.496444  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:28.821872  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.563208  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.862840  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.822223  9119 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:04:28.863453  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.822904  9269 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:28.822178  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:28.824949  9087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.001s	sys 0.023s
W20260709 13:04:28.864851  9087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.003s	sys 0.024s
W20260709 13:04:28.865381  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 681259us
I20260709 13:04:28.809573  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 453) took 1698 ms. Trace:
I20260709 13:04:28.867116  9120 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:04:28.824812  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.820840  9191 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:04:28.869141  9191 pending_rounds.cc:85] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 5
I20260709 13:04:28.869457  9191 pending_rounds.cc:107] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:04:28.869880  9191 raft_consensus.cc:2889] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:28.810379  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2017) took 2150 ms. Trace:
W20260709 13:04:28.813624 11863 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.395s	user 0.000s	sys 0.001s
I20260709 13:04:28.490021  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:28.866034  9087 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.696s (SENT)
W20260709 13:04:28.556135  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 61744.1us
W20260709 13:04:28.928125  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:28.973246  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 107253us
W20260709 13:04:28.973903  9087 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.704s (SENT)
I20260709 13:04:28.867024  9196 rpcz_store.cc:276] 0709 13:04:27.111079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.178064 (+ 66985us) service_pool.cc:224] Handling call
0709 13:04:28.809305 (+1631241us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b5400143554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:28.809495 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1763896064}
W20260709 13:04:28.872633  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 469201us
W20260709 13:04:28.888455  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:28.974838  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:28.411299 11858 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:28.870525  9190 rpcz_store.cc:276] 0709 13:04:27.732212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.030835 (+298623us) service_pool.cc:224] Handling call
0709 13:04:28.032035 (+  1200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.212634 (+180599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.811870 (+599236us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400398354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:28.812074 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283541760}
W20260709 13:04:28.975301  9238 leader_election.cc:336] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.203s (SENT)
I20260709 13:04:28.977192  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:28.977707  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:28.974493  9087 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.732s (SENT)
I20260709 13:04:28.975703  9857 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:28.972105  9117 rpcz_store.cc:276] 0709 13:04:26.659489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.659645 (+   156us) service_pool.cc:224] Handling call
0709 13:04:26.694112 (+ 34467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.110863 (+1416751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.810087 (+699224us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54001e4454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:28.810285 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3256847104}
I20260709 13:04:28.977846  9270 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:28.889235  9853 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:04:28.437008 11791 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:28.984058  9268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.006s	sys 0.010s
W20260709 13:04:28.727102 11870 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.510s	user 0.001s	sys 0.000s
I20260709 13:04:28.441135 11773 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:28.991083  9862 raft_consensus.cc:2368] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7.
I20260709 13:04:28.436758 11789 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:28.989960  9268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.006s	sys 0.010s
W20260709 13:04:29.037041  9825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 2.214s (SENT) [suppressed 48 similar messages]
W20260709 13:04:29.050705  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 76239.6us
I20260709 13:04:28.473562 11813 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:29.051267  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 76860.7us
I20260709 13:04:28.416273 11826 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.557s
I20260709 13:04:28.490504 11815 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:28.823621 11864 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.006s	sys 0.013s
I20260709 13:04:28.722573 11874 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.860579 11640 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:28.506769 11783 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:04:28.989735  9198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1863) took 1749 ms (client timeout 1753 ms). Trace:
I20260709 13:04:29.083458  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:28.562465 11788 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:28.723843 11800 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:28.867205 11769 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:28.860491 11801 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.060981  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:29.061568  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 196255us
I20260709 13:04:28.888231 11775 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:28.860534 11799 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:29.066809  9120 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:28.973769 11827 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:28.861256 11586 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:28.431617 11333 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.619s
W20260709 13:04:29.072659  9271 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.000s	sys 0.043s
W20260709 13:04:29.152236  9271 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.000s	sys 0.044s
I20260709 13:04:28.976889 11635 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:28.988711 11798 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.000s	sys 0.015s
W20260709 13:04:29.155231 11798 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.000s	sys 0.016s
I20260709 13:04:29.077616  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:29.157460  9270 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:29.155231 11635 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.078100  9117 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:29.160799  9115 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:29.162371 11790 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:28.979207 11779 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:29.081604  9859 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:28.431550 11869 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.648s
I20260709 13:04:29.054049 11833 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:29.082795  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 107531us
W20260709 13:04:29.059515  9087 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.752s (SENT)
W20260709 13:04:29.169643  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 110145us
I20260709 13:04:29.170096  9271 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:04:29.170364  9087 leader_election.cc:336] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.908s (SENT)
I20260709 13:04:29.170938  9087 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:29.172299  9087 leader_election.cc:336] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.872s (SENT)
I20260709 13:04:29.102726 11800 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:29.108115  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.176466  9162 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:29.077827 11864 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.012s	sys 0.013s
I20260709 13:04:29.176738  9121 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:29.084425 11874 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:29.084118  9123 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:29.114300 11839 consensus_meta.cc:287] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.883s	user 0.001s	sys 0.008s
I20260709 13:04:29.114874  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2035) took 2335 ms. Trace:
I20260709 13:04:29.077808 11881 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.118701 11793 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:29.208339  9194 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:29.209404  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.121076  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:29.083823  9198 rpcz_store.cc:269] 0709 13:04:27.240609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.282804 (+ 42195us) service_pool.cc:224] Handling call
0709 13:04:28.989600 (+1706796us) spinlock_profiling.cc:243] Waited 82.1 ms on lock 0x7b54000eef54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:28.989684 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188912000}
I20260709 13:04:29.072641 11858 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.215821 11531 consensus_queue.cc:1059] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:04:29.176326 11779 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.216676 11531 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.216271 11858 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.217116 11531 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.217164 11800 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.218400 11800 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.218940 11858 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:29.219324 11840 consensus_meta.cc:287] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.932s	user 0.008s	sys 0.002s
W20260709 13:04:29.139129  9089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.034s	sys 0.033s
I20260709 13:04:29.064960 11773 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:28.557883 11843 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.183111  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:29.185568  9272 rpcz_store.cc:276] 0709 13:04:26.779098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.784250 (+  5152us) service_pool.cc:224] Handling call
0709 13:04:26.784552 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.108521 (+1323969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.087097 (+978576us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54003039d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:29.114785 (+ 27688us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2791168896}
I20260709 13:04:29.202414 11586 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:04:29.178680 11866 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:04:29.213582  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:29.225941  9019 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.016s	sys 0.018s
W20260709 13:04:29.226627  9019 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.016s	sys 0.018s
I20260709 13:04:29.226034  9861 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:29.148123 11769 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.215987 11728 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:29.124169 11640 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:29.219978  9089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.034s	sys 0.033s
I20260709 13:04:29.219723  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.237017  9857 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:04:29.237323  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 1.06969e+06us
W20260709 13:04:29.147679  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 285049us
I20260709 13:04:29.237344  9853 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:29.238125  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.238786  9857 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:29.220619  9266 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:04:29.140866 11871 log.cc:927] Time spent T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:04:29.216014  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:29.242175  9198 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:29.221197  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44841.9us
I20260709 13:04:29.243496  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:29.220645  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.223409 11881 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:29.207988 11805 consensus_meta.cc:287] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.331s	user 0.009s	sys 0.003s
W20260709 13:04:29.233596  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 50602.3us
I20260709 13:04:29.250264  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:29.144860 11801 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:29.228490 11879 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:29.289649 11879 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.290071 11879 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:29.238155  9826 leader_election.cc:336] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.562s (SENT)
I20260709 13:04:29.244441  9854 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
W20260709 13:04:29.244604  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 68830.5us
I20260709 13:04:29.294430  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:29.294808  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56685us
I20260709 13:04:29.245124  9089 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:29.295146  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:29.295715  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 50664.2us
I20260709 13:04:29.295832  9854 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:29.296051  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.206636 11679 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:29.297116  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:29.207839 11635 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:29.222458  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.297333 11679 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.241058  9857 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:29.298612  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.299507 11801 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:29.299654  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:29.219341 11531 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:29.300505  9857 raft_consensus.cc:2410] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:29.296867 11712 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:29.301435  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:29.296830  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.298717  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.301957 11825 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.302919  9194 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:29.303177 11825 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:29.303637  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.304188  9194 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:29.304435  9164 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:29.304944  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:29.305068 11679 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:29.305853  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:29.306067  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:29.302218  9854 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:29.306545  9194 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:29.307260  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:29.262549 11769 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:29.242671 11865 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:04:29.308089 11869 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:29.308349 11635 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.308804  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1912) took 1461 ms. Trace:
W20260709 13:04:29.309041  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.307693 11783 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:29.309783 11869 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:29.307710  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:29.264599 11886 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:29.264650 11773 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:29.266347 11793 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:29.313681 11793 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.309185  9116 rpcz_store.cc:276] 0709 13:04:27.847427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.848177 (+   750us) service_pool.cc:224] Handling call
0709 13:04:27.865963 (+ 17786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.896533 (+ 30570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.896563 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.914203 (+ 17640us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.149725 (+235522us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:28.261245 (+111520us) log.cc:844] Serialized 14 byte log entry
0709 13:04:28.261459 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.261537 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.308617 (+1047080us) raft_consensus.cc:1685] finished
0709 13:04:29.308635 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.308749 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":657157,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":72,"raft.queue_time_us":359825,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":105468,"thread_start_us":275207,"threads_started":2,"wal-append.queue_time_us":815074}
I20260709 13:04:29.267133 11775 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:29.269073  9196 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:04:29.316004  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1883) took 1377 ms. Trace:
I20260709 13:04:29.316730 11826 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:29.241485 11728 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:29.317140  9238 connection.cc:256] RPC call timeout handler was delayed by -1.9800241850000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:04:29.264065  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.282331 11640 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:29.283525  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 585) took 2034 ms. Trace:
I20260709 13:04:29.311509 11869 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:29.340708  9264 rpcz_store.cc:276] 0709 13:04:27.248876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.823308 (+574432us) service_pool.cc:224] Handling call
0709 13:04:29.283372 (+1460064us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54000fa5d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:29.283471 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719814912}
I20260709 13:04:29.284804  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2049) took 1051 ms. Trace:
I20260709 13:04:29.288918 11874 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:29.239850 11840 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:29.294891  9087 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.090s (SENT)
I20260709 13:04:29.206229 11839 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.300444  9195 raft_consensus.cc:2410] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:29.307384  9267 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:29.307796  9858 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:29.311079 11818 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:29.311888 11776 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.189849 11833 consensus_queue.cc:1059] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 13:04:29.317801  9238 leader_election.cc:336] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.365s (SENT)
I20260709 13:04:29.356487  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:29.282912 11836 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:29.318378  9854 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.316020 11775 consensus_queue.cc:1243] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:04:29.301977  9857 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:29.260962 11858 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:29.311868  9089 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:29.347922 11568 consensus_meta.cc:287] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.858s	user 0.007s	sys 0.003s
W20260709 13:04:29.295838  9826 leader_election.cc:336] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.701s (SENT)
W20260709 13:04:29.364689  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 68873.9us
I20260709 13:04:29.258563  9198 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
W20260709 13:04:29.328841  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.332111  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.342172  9266 rpcz_store.cc:276] 0709 13:04:28.233476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.233727 (+   251us) service_pool.cc:224] Handling call
0709 13:04:28.252030 (+ 18303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.240678 (+988648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.284588 (+ 43910us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b54002a3854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:29.284747 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2226832640}
I20260709 13:04:29.316500  9192 rpcz_store.cc:276] 0709 13:04:27.938995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.135309 (+196314us) service_pool.cc:224] Handling call
0709 13:04:28.135645 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:28.155164 (+ 19519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:28.155196 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:28.170073 (+ 14877us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.431674 (+261601us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:28.431725 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.431811 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.315801 (+883990us) raft_consensus.cc:1685] finished
0709 13:04:29.315820 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.315951 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11697,"raft.queue_time_us":32422,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":16854,"thread_start_us":205990,"threads_started":1,"wal-append.queue_time_us":1116548}
I20260709 13:04:29.319597  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:29.237676 11873 raft_consensus.cc:3055] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:29.327430  9269 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:29.347836  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:29.297448 11881 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:29.315990 11834 consensus_meta.cc:287] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.122s	user 0.003s	sys 0.007s
W20260709 13:04:29.344588 11808 consensus_meta.cc:287] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.093s	user 0.009s	sys 0.002s
I20260709 13:04:29.315642 11793 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:29.316753 11872 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:29.362180  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1671) took 2085 ms. Trace:
I20260709 13:04:29.307405  9194 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 2. Current term is 3. Ops: []
W20260709 13:04:29.362567  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 119148us
I20260709 13:04:29.345408 11769 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.400450  9265 rpcz_store.cc:276] 0709 13:04:27.277179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.827352 (+550173us) service_pool.cc:224] Handling call
0709 13:04:27.827655 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:27.921034 (+ 93379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:27.921064 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:27.921585 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:04:28.205722 (+284137us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:28.205776 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:28.205871 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.361981 (+1156110us) raft_consensus.cc:1685] finished
0709 13:04:29.361999 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.362124 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16481,"raft.queue_time_us":25998,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":286286,"threads_started":2,"wal-append.queue_time_us":663298}
W20260709 13:04:29.367820  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48294.5us
I20260709 13:04:29.404999  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:29.368213  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 61006.3us
W20260709 13:04:29.371726  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 59927.1us
I20260709 13:04:29.407286  9089 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:29.376292  9190 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
W20260709 13:04:29.376559  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 81695.4us
I20260709 13:04:29.379396  9861 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:04:29.408685  9087 leader_election.cc:336] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.142s (SENT)
I20260709 13:04:29.408938  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:29.409206  9087 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.142s (SENT)
I20260709 13:04:29.409629  9861 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:29.409595  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:29.410449  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.382189  9854 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:29.383438  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:29.383832  9826 leader_election.cc:336] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.235s (SENT)
I20260709 13:04:29.413094  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:29.347201 11869 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.385483  9858 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:04:29.351482 11840 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:29.390640  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 781) took 2273 ms. Trace:
I20260709 13:04:29.414016 11869 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:29.414217 11840 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:29.359088 11754 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.363624 11712 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:29.376340 11891 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.382802 11805 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:29.380225 11640 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.391775 11779 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.376014 11531 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:29.392776 11727 consensus_meta.cc:287] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.263s	user 0.011s	sys 0.000s
I20260709 13:04:29.372454 11873 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:29.380681 11483 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:04:29.376932 11874 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:29.417060  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.417274  9857 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:29.414026  9858 pending_rounds.cc:85] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 6
I20260709 13:04:29.438102  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.421607  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:29.422012  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:29.438280  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:29.438133  9858 pending_rounds.cc:107] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.7
I20260709 13:04:29.439604  9858 raft_consensus.cc:2917] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Aborting config change with OpId 7.7: Aborted: Op aborted by new leader
I20260709 13:04:29.440193  9858 raft_consensus.cc:2889] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:29.440266  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:29.420104 11783 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:29.441349 11783 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:29.441975 11783 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.423099 11889 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:29.376204 11679 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.361035 11776 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:29.431447 11900 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:04:29.452176 11893 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:29.456220  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.456830  9860 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:29.457669  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:29.458287  9860 raft_consensus.cc:2368] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4.
I20260709 13:04:29.414233  9859 rpcz_store.cc:276] 0709 13:04:27.116988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.117762 (+   774us) service_pool.cc:224] Handling call
0709 13:04:27.118030 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.163626 (+2045596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.390281 (+226655us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b5400657a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:29.390558 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4515583232}
I20260709 13:04:29.426055 11586 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:04:29.459719 11725 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:29.460175 11725 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:29.460379  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:29.415576 11883 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.437594  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.438397  9853 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:29.377077 11568 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.438225  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:29.437727  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.465920  9089 connection.cc:256] RPC call timeout handler was delayed by -1.7670491339999899s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 13:04:29.466331  9089 connection.cc:256] RPC call timeout handler was delayed by -1.7166491340000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
W20260709 13:04:29.466616  9089 connection.cc:256] RPC call timeout handler was delayed by -1.5933491339999932s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
W20260709 13:04:29.466912  9089 connection.cc:256] RPC call timeout handler was delayed by -1.1919491340000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 13:04:29.417564  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 56295.5us
I20260709 13:04:29.421015 11805 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.417723 11881 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.415410 11754 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:29.475435 11568 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:29.454200 11836 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:04:29.480343 11855 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.480633  9117 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:29.481195 11805 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:29.481649 11867 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:04:29.482170  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2046) took 1260 ms. Trace:
I20260709 13:04:29.418702 11864 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:04:29.482512  9267 rpcz_store.cc:276] 0709 13:04:28.221706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.222632 (+   926us) service_pool.cc:224] Handling call
0709 13:04:28.223011 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.352374 (+1129363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.481447 (+129073us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b54001edfa0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f765665bdc0 00007f76566e7a02 00007f76566e3dd6 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:29.481958 (+   511us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400303ed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:29.482116 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2497915008}
I20260709 13:04:29.427054 11891 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.474726  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:29.484335  9195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1867) took 2242 ms (client timeout 2142 ms). Trace:
I20260709 13:04:29.484561  9856 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
W20260709 13:04:29.484719  9195 rpcz_store.cc:269] 0709 13:04:27.241557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.707193 (+465636us) service_pool.cc:224] Handling call
0709 13:04:29.484135 (+1776942us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b540010b654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:29.484253 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3632298880}
I20260709 13:04:29.485603  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.485857  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.486943  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.488905  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.489462  9115 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:29.490397  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.392524 11808 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:29.419426 11872 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:29.426235 11333 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:29.428573 11531 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.515210 11788 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:29.515193 11725 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.516700 11873 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:29.518579 11873 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:29.520252  9087 connection.cc:256] RPC call timeout handler was delayed by -1.2015892809999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
I20260709 13:04:29.520547 11891 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:29.439018 11806 consensus_meta.cc:287] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.650s	user 0.012s	sys 0.001s
I20260709 13:04:29.436929  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.461887  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:29.464773  9859 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:29.453876  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.528573  9860 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:29.528991  9114 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:29.531488  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.533383  9859 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:29.474086  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:29.534348  9119 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.535115  9859 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:29.474671 11884 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:29.536942  9193 raft_consensus.cc:606] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:29.537569  9193 raft_consensus.cc:614] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:29.482683 11867 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:29.445629 11793 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:29.446563 11834 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.443092 11874 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.493325  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.419500 11769 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.453682 11813 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:29.500962  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:29.507712  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:29.510748  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1872) took 2241 ms. Trace:
I20260709 13:04:29.511898 11333 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.480657 11886 raft_consensus.cc:3055] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:29.515504 11833 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.515995 11531 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:29.521193  9087 leader_election.cc:336] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.589s (SENT)
I20260709 13:04:29.560946  9087 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:29.453737 11779 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:29.523236  9235 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:29.428136 11640 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.561748  9854 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:29.532133  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.538058  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:29.480885 11855 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.538673  9189 raft_consensus.cc:2410] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:04:29.473316  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.542034 11793 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:29.539626  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 71614.9us
I20260709 13:04:29.564240  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:29.564697  9087 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:29.565019 11727 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:29.565377  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:29.565984  9117 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.566814  9089 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:29.540663  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.568044  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:29.568464 11635 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:29.569367  9123 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:29.569759  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:46626 (request call id 18) took 1768 ms. Trace:
I20260709 13:04:29.570137  9193 rpcz_store.cc:276] 0709 13:04:27.801587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:27.904567 (+102980us) service_pool.cc:224] Handling call
0709 13:04:29.569571 (+1665004us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b5400421dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566de853 00007f765b6ac60a 00007f7654c327d4 00007f7654c32762 00007f7654c32691 00007f7654c3260c 00007f7654c318c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:29.569676 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3475907968}
I20260709 13:04:29.496886 11790 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.542944  9195 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
W20260709 13:04:29.544154  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 107812us
I20260709 13:04:29.513365  9825 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:29.548204  9859 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:29.465337 11679 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.557170 11586 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:04:29.548763  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 47925.8us
I20260709 13:04:29.498639 11774 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 13:04:29.524501  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.566560 11805 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.568140 11912 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:29.542379  9122 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:29.573264  9862 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 790) took 2836 ms (client timeout 2172 ms). Trace:
I20260709 13:04:29.540263  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:29.540768 11867 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:29.681025  9194 raft_consensus.cc:2368] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4.
I20260709 13:04:29.544315 11874 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:29.558245 11911 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:04:29.549548 11775 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.580197  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.696297  9114 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:29.588676 11893 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:29.582808  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:29.585980  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:29.586195  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 72946.9us
I20260709 13:04:29.587239 11776 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:29.703169  9269 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:29.587423  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.587777  9265 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:04:29.592998 11813 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:29.589982  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:29.548722  9191 rpcz_store.cc:276] 0709 13:04:27.269355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.028199 (+758844us) service_pool.cc:224] Handling call
0709 13:04:28.820401 (+792202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:28.973971 (+153570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.510491 (+536520us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b5400143550. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:29.510670 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1821245824}
I20260709 13:04:29.556864 11886 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:04:29.592953  9862 rpcz_store.cc:269] 0709 13:04:26.736751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.739343 (+  2592us) service_pool.cc:224] Handling call
0709 13:04:29.573036 (+2833693us) spinlock_profiling.cc:243] Waited 2.24 s on lock 0x7b54002e9d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:29.573179 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5149828864}
I20260709 13:04:29.562939 11817 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.665117 11586 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:29.664907  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:29.681257 11912 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:29.571557 11796 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:29.574198 11889 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.548082 11834 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:29.660902  9117 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:29.682670  9856 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:29.702240 11791 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.584977 11779 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:29.686290  9194 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:29.556813 11333 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.697697  9121 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:04:29.733145  9121 pending_rounds.cc:85] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 5
I20260709 13:04:29.733393  9121 pending_rounds.cc:107] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:04:29.733620  9121 raft_consensus.cc:2889] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:29.588419  9266 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
W20260709 13:04:29.707144  9825 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.549s (SENT)
I20260709 13:04:29.734277  9266 pending_rounds.cc:85] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 6
I20260709 13:04:29.734589  9266 pending_rounds.cc:107] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.7
I20260709 13:04:29.734845  9266 raft_consensus.cc:2917] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Aborting config change with OpId 7.7: Aborted: Op aborted by new leader
I20260709 13:04:29.735325  9266 raft_consensus.cc:2889] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:29.707942  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.703238  9264 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
W20260709 13:04:29.711658  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 121763us
I20260709 13:04:29.714598 11836 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.712782  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:29.714762 11846 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:29.681027 11827 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:29.663322  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 125303us
W20260709 13:04:29.713864  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 147116us
I20260709 13:04:29.573460 11793 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:29.563460 11833 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.685107 11801 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:29.713374 11908 log.cc:927] Time spent T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:04:29.588403 11790 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.749161  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:29.735052 11894 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:04:29.837320 11919 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:29.749245  9859 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
W20260709 13:04:29.838863  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 88980.4us
I20260709 13:04:29.587584 11872 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:29.839305  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 85220.7us
I20260709 13:04:29.724814 11796 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:29.840245  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:29.840548 11796 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.843343 11856 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:29.728466 11826 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:29.773751  9826 leader_election.cc:336] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.615s (SENT)
I20260709 13:04:29.846536  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:29.836951 11914 log.cc:927] Time spent T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:04:29.838996  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2038) took 1675 ms. Trace:
I20260709 13:04:29.717298 11817 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:29.880756  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.892354  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 137122us
I20260709 13:04:29.892396  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.893673  9116 raft_consensus.cc:2410] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:29.850090  9268 rpcz_store.cc:276] 0709 13:04:28.163113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.163238 (+   125us) service_pool.cc:224] Handling call
0709 13:04:28.163587 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:29.057818 (+894231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.057847 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:29.058345 (+   498us) log.cc:844] Serialized 52 byte log entry
0709 13:04:29.226572 (+168227us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:29.226608 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.361818 (+135210us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b540032e954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:29.361880 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:29.838660 (+476780us) raft_consensus.cc:1685] finished
0709 13:04:29.838684 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:29.838935 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22474,"raft.queue_time_us":46843,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":7667,"spinlock_wait_cycles":4713728,"thread_start_us":1011721,"threads_started":2,"wal-append.queue_time_us":335063}
I20260709 13:04:29.751791 11827 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:29.900008  9089 leader_election.cc:400] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4.
W20260709 13:04:29.900175  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 126481us
I20260709 13:04:29.906008  9089 leader_election.cc:403] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:04:29.732396 11791 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.835422 11879 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.844058 11912 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.926903 11917 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 13:04:29.948495 11916 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:04:29.913964 11827 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:29.914244 11846 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:29.952214 11846 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:29.736379 11909 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.170s
W20260709 13:04:29.934813  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:29.976627  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:29.943274  9861 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:29.931875  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:29.986768  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55019.4us
W20260709 13:04:29.992972  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 151327us
I20260709 13:04:29.906488 11635 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:29.763964 11836 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:29.995225  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 246150us
W20260709 13:04:29.995810  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 158904us
I20260709 13:04:29.997326  9119 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:29.905390 11891 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:29.987884  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:29.831902 11801 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.944789 11834 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:29.942881 11879 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:29.996145 11889 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:30.021370  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:29.997076 11800 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4.
I20260709 13:04:29.913511 11886 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.098856 11776 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:30.094013 11727 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:04:30.101087 11839 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.004346 11769 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:30.112237  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.093662 11923 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:29.822642 11867 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.097831 11889 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.103982 11640 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:29.905117 11826 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:29.906778 11813 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.105868 11917 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:30.115072  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 93790.1us
I20260709 13:04:30.131695  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:29.953217 11586 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.122602 11834 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:30.164175 11923 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.165393 11846 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:30.133051 11889 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:30.130055 11805 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.200048  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.207937  9119 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
W20260709 13:04:30.180563  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:30.167353 11840 consensus_meta.cc:287] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.752s	user 0.008s	sys 0.001s
I20260709 13:04:30.210397  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.161978 11886 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:30.012753 11779 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.214329  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.176177 11833 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.217398  9268 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:30.210760  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:30.224495  9271 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:30.172850 11801 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.178284 11917 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.198792 11834 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.174180 11836 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.221879  9119 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:30.218143 11931 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.212941 11923 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:30.227078  9162 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:30.228657  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.237601  9118 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:30.212064 11839 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:04:30.216199 11773 raft_consensus.cc:3055] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:30.178134 11835 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:04:30.214011 11846 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:04:30.232048 11917 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:30.221035 11840 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.251240 11931 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.255702  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.248322  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:30.272111  9119 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:30.259684  9271 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:30.234472 11834 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:30.270460  9267 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.002s	sys 0.008s
W20260709 13:04:30.230667  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 223310us
W20260709 13:04:30.259158 11874 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.000s
I20260709 13:04:30.252844 11790 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:30.277216  9267 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.002s	sys 0.008s
I20260709 13:04:30.350824 11808 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:30.351493 11881 consensus_meta.cc:287] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.875s	user 0.010s	sys 0.000s
I20260709 13:04:30.337137 11840 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.351972 11881 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:30.352945  9115 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:30.250648 11836 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:30.346172  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.358139  9118 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:30.259851 11791 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.306484  9119 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
W20260709 13:04:30.356084  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 58335.7us
I20260709 13:04:30.253384 11773 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:30.359551  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:30.291383  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 75924.8us
W20260709 13:04:30.338984 11873 consensus_meta.cc:287] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.820s	user 0.003s	sys 0.009s
W20260709 13:04:30.342823 11793 consensus_meta.cc:287] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.001s
W20260709 13:04:30.343484 11333 consensus_meta.cc:287] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.000s
W20260709 13:04:30.332628 11932 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:04:30.365828  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:30.312580 11783 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.864s	user 0.009s	sys 0.001s
I20260709 13:04:30.252729 11796 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:30.339419 11834 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:30.332058 11935 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
W20260709 13:04:30.383733  9089 leader_election.cc:336] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.142s (SENT)
I20260709 13:04:30.372957  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:30.363636 11808 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.273334 11827 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.409020  9855 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:04:30.526643  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 144403us
I20260709 13:04:30.375373  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:30.560859  9263 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:30.576503  9119 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:30.384120  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:30.575695  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 470) took 1025 ms. Trace:
I20260709 13:04:30.439350  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:30.577844  9089 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:30.582201  9860 raft_consensus.cc:2410] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:04:30.581494  9270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.006s	sys 0.018s
W20260709 13:04:30.585741  9270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.007s	sys 0.019s
I20260709 13:04:30.357825 11836 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:30.481693  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.585831  9118 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:30.586354  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 202400us
I20260709 13:04:30.588881  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:30.589843  9119 raft_consensus.cc:2376] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4. Re-sending same reply.
I20260709 13:04:30.579068  9193 rpcz_store.cc:276] 0709 13:04:29.550404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.716163 (+165759us) service_pool.cc:224] Handling call
0709 13:04:29.717432 (+  1269us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:29.821895 (+104463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.822018 (+   123us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:04:29.843904 (+ 21886us) log.cc:844] Serialized 78 byte log entry
0709 13:04:30.293290 (+449386us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:30.336006 (+ 42716us) log.cc:844] Serialized 14 byte log entry
0709 13:04:30.354034 (+ 18028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.354119 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.442212 (+ 88093us) raft_consensus.cc:1685] finished
0709 13:04:30.442232 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:30.575626 (+133394us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41775,"prepare.queue_time_us":51911,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":114403,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":246,"thread_start_us":448989,"threads_started":1,"wal-append.queue_time_us":595755}
I20260709 13:04:30.591970  9089 leader_election.cc:400] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4.
W20260709 13:04:30.574193 11856 consensus_meta.cc:287] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.730s	user 0.002s	sys 0.006s
I20260709 13:04:30.592360  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:30.414469 11928 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
W20260709 13:04:30.411657 11869 consensus_meta.cc:287] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.996s	user 0.007s	sys 0.003s
I20260709 13:04:30.560436 11796 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:30.365149 11679 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:30.410121 11531 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.832s	user 0.009s	sys 0.001s
I20260709 13:04:30.595372  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.567849 11834 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:30.607039  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.367267 11874 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.608093  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.648758  9269 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:30.649746  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.650411  9269 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:30.651172  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.607414  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.608876  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.611163  9860 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:30.412072 11783 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:30.611681  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:30.443249 11790 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:30.659231  9269 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:30.415448 11917 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.641808  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.642851  9267 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:30.558319 11889 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.667171  9191 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:30.591107 11836 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:30.603673 11800 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:04:30.647037  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.441744 11931 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.571393 11923 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:30.654632  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.443127 11858 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.198s
I20260709 13:04:30.656318  9272 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:30.656322  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.607362 11856 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:30.638175 11791 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:30.436240 11333 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:30.660079  9189 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1865) took 3800 ms (client timeout 1577 ms). Trace:
I20260709 13:04:30.382262 11793 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:30.593674  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 230602us
W20260709 13:04:30.728667  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:30.666451  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:30.588886 11915 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:30.730310  9267 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:04:30.730464 11915 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:30.731038 11915 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:30.589483 11808 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.734210 11808 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.734692 11808 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:30.735159 11808 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:30.670701  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55266.3us
I20260709 13:04:30.661729 11790 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:30.695688  9189 rpcz_store.cc:269] 0709 13:04:26.855344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.855523 (+   179us) service_pool.cc:224] Handling call
0709 13:04:30.651422 (+3795899us) spinlock_profiling.cc:243] Waited 2.68 s on lock 0x7b5400421dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:30.655440 (+  4018us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6162112896}
I20260709 13:04:30.743049  9859 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:30.701165  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.684165  9116 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:30.702644  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.744714  9272 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:30.745658  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:30.438071 11774 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:30.704046  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.703483  9265 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 2.
I20260709 13:04:30.747370  9267 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:30.713059  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "991b28944986479a8ad096ca92f1a435"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:30.748159  9193 raft_consensus.cc:606] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:30.748612  9193 raft_consensus.cc:614] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:30.713862  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:30.722698  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1934) took 1273 ms. Trace:
I20260709 13:04:30.652578 11531 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.726468  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.727232  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:30.605382 11881 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:04:30.637439 11869 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:30.742039  9089 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:30.670481 11779 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:04:30.709308 11858 consensus_queue.cc:1243] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 94504d29c6744f38bb06943479df9466 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:04:30.749925  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.951901  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.952488  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.953100  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.953260  9267 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:30.954139  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:30.954344  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.955148  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:30.956082  9198 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:30.758538  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.757937  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:30.957356  9265 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:30.988101  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:31.066293  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 324298us
I20260709 13:04:31.067000  9189 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:30.988480  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:30.750116  9271 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:31.003758  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:31.001608  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:31.071385  9195 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:31.071417  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:31.071967  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:31.070854  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2068) took 1480 ms. Trace:
I20260709 13:04:31.072444  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 841) took 1621 ms. Trace:
I20260709 13:04:31.073313  9861 rpcz_store.cc:276] 0709 13:04:29.450874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.450972 (+    98us) service_pool.cc:224] Handling call
0709 13:04:29.943010 (+492038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.985919 (+ 42909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.072268 (+1086349us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54007382d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:31.072393 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1130922752}
I20260709 13:04:31.073063  9270 rpcz_store.cc:276] 0709 13:04:29.590918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.591059 (+   141us) service_pool.cc:224] Handling call
0709 13:04:29.665708 (+ 74649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.586251 (+920543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.586278 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:30.586728 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:04:30.709639 (+122911us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:30.709700 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.955595 (+245895us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54001c5cd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:30.955641 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.955688 (+    47us) raft_consensus.cc:1685] finished
0709 13:04:30.955700 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.070750 (+115050us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":302671,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":41994,"spinlock_wait_cycles":431884032,"thread_start_us":814551,"threads_started":2,"wal-append.queue_time_us":354871}
I20260709 13:04:30.984568  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:30.825423  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:31.070870  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:30.990988  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:30.957057  9121 rpcz_store.cc:276] 0709 13:04:29.449376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.453187 (+  3811us) service_pool.cc:224] Handling call
0709 13:04:29.453531 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:29.734057 (+280526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:30.722367 (+988310us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540023cdd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:30.722616 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560528000}
I20260709 13:04:30.970606  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:31.018157  9119 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:31.110703  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 440) took 1093 ms. Trace:
I20260709 13:04:31.112516  9119 rpcz_store.cc:276] 0709 13:04:30.016895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.954949 (+938054us) service_pool.cc:224] Handling call
0709 13:04:31.110638 (+155689us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.114004  9120 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
W20260709 13:04:30.799499  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 112594us
I20260709 13:04:31.082365  9192 raft_consensus.cc:2410] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:31.115712  9268 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:31.116153  9267 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:31.116644  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:31.116964  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:31.117265  9268 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:31.118115  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:31.119324  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:31.072490  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:31.120116  9272 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:30.821945 11915 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.120740 11915 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.121004  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:31.121194 11915 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.121927  9272 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:31.118146  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:31.070345  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:30.841666 11925 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.721s	user 0.000s	sys 0.003s
I20260709 13:04:31.124087  9268 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:30.747625 11790 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:31.136524  9272 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:04:30.821139 11946 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:04:31.114212  9238 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:30.794278 11825 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:31.055208 11834 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.140661  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1942) took 1141 ms. Trace:
I20260709 13:04:31.247129  9121 rpcz_store.cc:276] 0709 13:04:29.999603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.111515 (+1111912us) service_pool.cc:224] Handling call
0709 13:04:31.140044 (+ 28529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.140262 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.140299 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.140354 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:31.140405 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.140475 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.140593 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.250593  9121 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 13:04:31.251531  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 439) took 1234 ms. Trace:
I20260709 13:04:31.252259  9121 rpcz_store.cc:276] 0709 13:04:30.016673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.249297 (+1232624us) service_pool.cc:224] Handling call
0709 13:04:31.250039 (+   742us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:31.251132 (+  1093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.251425 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:31.242965  9087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.028s	sys 0.069s
I20260709 13:04:31.138228  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1941) took 1304 ms. Trace:
W20260709 13:04:31.256134  9087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.029s	sys 0.070s
I20260709 13:04:31.256445  9114 rpcz_store.cc:276] 0709 13:04:29.834276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.113573 (+1279297us) service_pool.cc:224] Handling call
0709 13:04:31.137737 (+ 24164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.137913 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.137939 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.137978 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:31.138013 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.138079 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.138178 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.255570  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 441) took 1238 ms. Trace:
I20260709 13:04:31.245246  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2021) took 1777 ms. Trace:
I20260709 13:04:31.259420  9114 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:31.258787  9121 rpcz_store.cc:276] 0709 13:04:30.017158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.253923 (+1236765us) service_pool.cc:224] Handling call
0709 13:04:31.254538 (+   615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.255078 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.255149 (+    71us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.255225 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:31.255271 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.255339 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.255477 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.260433  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 444) took 1242 ms. Trace:
I20260709 13:04:31.245795  9118 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:31.261086  9114 rpcz_store.cc:276] 0709 13:04:30.017879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.258109 (+1240230us) service_pool.cc:224] Handling call
0709 13:04:31.258725 (+   616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.260107 (+  1382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.260364 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:31.261629  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 437) took 1245 ms. Trace:
I20260709 13:04:31.262431  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:31.259969  9123 rpcz_store.cc:276] 0709 13:04:29.467363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.570041 (+102678us) service_pool.cc:224] Handling call
0709 13:04:29.570334 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.570644 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.570669 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.571084 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 13:04:29.898553 (+327469us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:29.898607 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:29.898701 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:30.841041 (+942340us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:31.244040 (+402999us) raft_consensus.cc:1685] finished
0709 13:04:31.244131 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.244914 (+   783us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":445,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":14101760,"thread_start_us":327126,"threads_started":1,"wal-append.queue_time_us":431786}
I20260709 13:04:31.261967  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:31.262140  9118 rpcz_store.cc:276] 0709 13:04:30.016217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.137683 (+1121466us) service_pool.cc:224] Handling call
0709 13:04:31.245197 (+107514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.261347 (+ 16150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.261541 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:31.265553  9164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.008s	sys 0.011s
W20260709 13:04:31.266084  9164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.008s	sys 0.011s
I20260709 13:04:31.145889  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:31.268967  9194 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:31.271021  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:31.276926  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:31.278285  9194 raft_consensus.cc:2410] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:31.280349  9194 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:04:31.280601  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 166345us
I20260709 13:04:31.282052  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:31.287366  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:31.289729  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:31.172537  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2066) took 1586 ms. Trace:
I20260709 13:04:31.292661  9263 rpcz_store.cc:276] 0709 13:04:29.585615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.586528 (+   913us) service_pool.cc:224] Handling call
0709 13:04:29.586870 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.577941 (+991071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.172190 (+594249us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54004452d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:31.172458 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239327488}
I20260709 13:04:31.146627  9270 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:31.164418  9854 raft_consensus.cc:1217] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEADER]: Deduplicated request from leader. Original: 3.5->[5.6-5.6]   Dedup: 5.6->[]
I20260709 13:04:31.189327  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:31.301843  9854 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:04:31.186342  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 840) took 1735 ms. Trace:
I20260709 13:04:31.303557  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:31.305143  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:31.304054  9857 rpcz_store.cc:276] 0709 13:04:29.450631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.452104 (+  1473us) service_pool.cc:224] Handling call
0709 13:04:29.452363 (+   259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:29.492298 (+ 39935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:29.492327 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:29.499523 (+  7196us) log.cc:844] Serialized 26 byte log entry
0709 13:04:29.902552 (+403029us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:29.902606 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.176173 (+1273567us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b5400018bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:31.176239 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:31.176308 (+    69us) raft_consensus.cc:1685] finished
0709 13:04:31.176327 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:31.186247 (+  9920us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4568,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":10168,"spinlock_wait_cycles":91071872,"thread_start_us":361824,"threads_started":1,"wal-append.queue_time_us":718272}
I20260709 13:04:30.994001 11791 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:31.306532  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:31.167524  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:31.307622 11791 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.308961  9862 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:30.840148 11333 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:30.827399 11728 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.832331 11754 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:30.838364 11773 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.311264 11791 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.312186 11791 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.313148 11791 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:31.313330 11773 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:31.313560 11531 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.314214 11791 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:30.837707 11793 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:30.805693 11774 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:31.148651 11796 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:31.316540 11796 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.317102 11796 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:31.317984 11635 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:31.318475 11774 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.318619 11635 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:30.838847 11783 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.321580  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2024) took 1853 ms. Trace:
W20260709 13:04:31.322134  9115 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2026) took 1849 ms (client timeout 1659 ms). Trace:
I20260709 13:04:31.322016  9116 rpcz_store.cc:276] 0709 13:04:29.468094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.954629 (+486535us) service_pool.cc:224] Handling call
0709 13:04:29.990229 (+ 35600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:30.744291 (+754062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.320927 (+576636us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b5400267354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:31.321356 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594961792}
W20260709 13:04:31.322749  9115 rpcz_store.cc:269] 0709 13:04:29.472605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.490175 (+ 17570us) service_pool.cc:224] Handling call
0709 13:04:31.321773 (+1831598us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b5400285d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:31.321934 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1982443264}
I20260709 13:04:31.303532  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:31.325089  9854 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:31.111227 11800 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:30.847254 11917 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.328804 11800 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.329429 11333 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.329831 11801 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:04:31.205519 11952 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:04:30.817859 11953 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.147351 11825 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:31.370378 11953 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:31.205525 11959 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:04:30.945173 11827 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.145545 11834 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:31.207810 11836 consensus_meta.cc:287] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.000s
I20260709 13:04:31.377790 11834 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:30.844175 11855 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.377281 11635 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.379138  9116 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:31.379335  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 652058us
I20260709 13:04:31.210700 11869 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.380584  9825 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:31.153154 11883 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.846457 11889 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.124233 11923 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.429075 11923 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:31.113873 11931 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.329699 11800 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:31.331233 11801 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:31.330252 11333 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.336592 11796 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.348176  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:31.350646  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:31.355579  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 469) took 1812 ms. Trace:
I20260709 13:04:31.350725 11962 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:04:31.361248  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1881) took 1897 ms. Trace:
I20260709 13:04:31.362807  9854 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:31.364202  9197 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:31.366832 11728 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.377236 11836 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:31.182153 11839 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.013s	sys 0.011s
I20260709 13:04:30.842617 11874 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.382522 11635 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.371516 11825 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.391767  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:31.393343 11640 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:31.393694 11790 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.395089 11679 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.395614  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:31.395906  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:31.398232  9825 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:31.398874 11725 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:31.413231  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:31.415382  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:31.416116  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:31.417755  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:31.424831  9087 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.545s (SENT)
I20260709 13:04:31.430876  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:31.583766  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 294242us
I20260709 13:04:31.624784  9118 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.610780  9862 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
W20260709 13:04:31.600642  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.528513  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:31.624490  9855 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:31.569797  9117 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:31.483179  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:31.611622  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 213603us
I20260709 13:04:31.610152  9857 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.611286  9115 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 2.
W20260709 13:04:31.622853  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 198157us
I20260709 13:04:31.623365  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1679) took 1001 ms. Trace:
W20260709 13:04:31.568728  9122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 1915) took 3161 ms (client timeout 3363 ms). Trace:
I20260709 13:04:31.429332 11826 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.534926 11889 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:31.528085  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 305094us
I20260709 13:04:31.609323  9191 rpcz_store.cc:276] 0709 13:04:29.543414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.711478 (+168064us) service_pool.cc:224] Handling call
0709 13:04:29.711855 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.667712 (+955857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.354828 (+687116us) spinlock_profiling.cc:243] Waited 955 ms on lock 0x7b5400398354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:31.355328 (+   500us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2196390144}
I20260709 13:04:31.596175 11976 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:31.600831 11858 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.625799 11775 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.626088 11333 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.608548  9195 rpcz_store.cc:276] 0709 13:04:29.463812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.587183 (+123371us) service_pool.cc:224] Handling call
0709 13:04:31.360687 (+1773504us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400423bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:31.361011 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3144627968}
W20260709 13:04:31.687625  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:31.614821 11918 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.416s	user 0.002s	sys 0.000s
I20260709 13:04:31.687233 11976 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:31.690080  9087 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.901s (SENT)
I20260709 13:04:31.619181 11640 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:31.626977 11790 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:31.683319  9122 rpcz_store.cc:269] 0709 13:04:28.374139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:28.405108 (+ 30969us) service_pool.cc:224] Handling call
0709 13:04:31.534782 (+3129674us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54002d32d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:31.535054 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559829888}
I20260709 13:04:31.682042  9269 rpcz_store.cc:276] 0709 13:04:30.621438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.650995 (+ 29557us) service_pool.cc:224] Handling call
0709 13:04:31.622901 (+971906us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b54000ebad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:31.623091 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17142016}
I20260709 13:04:31.695055  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:31.623935 11800 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.500761 11970 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.583292 11808 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.611557 11953 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.499938 11773 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.500424 11967 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.694646 11640 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.703915  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:31.693205 11976 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.624110 11679 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.715312 11977 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:31.708122 11728 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.716965  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:31.712844  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:31.717767  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:31.718799  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:31.718298 11773 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.711988 11775 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:31.596338  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:31.728219  9116 raft_consensus.cc:2376] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5. Re-sending same reply.
I20260709 13:04:31.431324 11883 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:31.707201 11640 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.730840  9193 raft_consensus.cc:606] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:31.709326 11790 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.688256 11931 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.726908 11800 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:31.744751  9189 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:31.735865  9193 raft_consensus.cc:614] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:31.746908  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:31.749991  9191 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
W20260709 13:04:31.747284 11839 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.017s	sys 0.014s
I20260709 13:04:31.755932 11980 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:04:31.758090  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 61197.3us
I20260709 13:04:31.732246 11889 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:31.753302 11977 consensus_queue.cc:1243] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer 94504d29c6744f38bb06943479df9466 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:31.756479 11775 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:31.768456  9271 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:31.772233 11801 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:31.773432  9195 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:31.779697  9121 raft_consensus.cc:2410] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:31.773862  9853 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:31.774359  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:31.776913 11836 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.777490 11796 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.778153 11889 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.778558 11923 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:31.791311  9189 raft_consensus.cc:2410] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:31.767624  9197 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.781214  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:31.781936 11775 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.782871 11801 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:31.785743  9265 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:31.787166 11915 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.789474 11728 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.790040 11980 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:31.734086 11858 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.780175  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:31.795670 11791 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.816696 11791 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.816789 11980 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.818989  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 606) took 1202 ms. Trace:
I20260709 13:04:31.819443  9271 rpcz_store.cc:276] 0709 13:04:30.617049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.070070 (+453021us) service_pool.cc:224] Handling call
0709 13:04:31.818743 (+748673us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54000ec4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:31.818903 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1468695808}
I20260709 13:04:31.744217 11967 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:31.815322  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:31.800588  9191 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:04:31.808060 11775 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.797068 11800 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:31.818933  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 66769.3us
I20260709 13:04:31.745797 11970 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:31.823431  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.827312 11679 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:31.827358 11969 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.831517  9118 raft_consensus.cc:2410] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:31.765734 11790 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:31.843550  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 67536.6us
I20260709 13:04:31.847536  9855 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:31.854914  9197 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:31.830747 11640 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.832636 11800 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.881270 11982 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:31.885713  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:31.876173  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 81080.4us
I20260709 13:04:31.884064  9825 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:31.878953  9235 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:31.867969 11769 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:31.870800 11915 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.903910  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:31.904325 11915 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.905301  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:31.906072  9122 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:31.906464 11915 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.882807 11970 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.884112 11858 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.885022 11775 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.886770 11874 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:31.888887 11640 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.886642 11982 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:31.892705 11773 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.907925 11769 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:31.879056 11801 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
W20260709 13:04:31.924793  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 217623us
I20260709 13:04:31.938508  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:31.934307 11923 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:31.940375 11923 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:31.940804 11923 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:31.934284 11874 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.944787 11982 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:31.946267 11874 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.947144 11640 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:31.948755 11640 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:31.949218 11640 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:31.952369 11790 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:31.955229  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:31.956179 11931 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:31.936944 11769 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:31.957008  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:31.957736 11883 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.957886  9123 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:31.892546 11969 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:31.967334 11969 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:31.956823 11967 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:31.934516 11970 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:31.977991  9857 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:31.968148  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:31.969604 11981 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:04:31.971661 11817 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:31.964830 11972 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:04:31.974793  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:31.966923 11976 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:31.977861  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 110812us
W20260709 13:04:31.981034  9856 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 850) took 2485 ms (client timeout 1928 ms). Trace:
W20260709 13:04:31.984977  9856 rpcz_store.cc:269] 0709 13:04:29.477783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.485304 (+  7521us) service_pool.cc:224] Handling call
0709 13:04:31.963307 (+2478003us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54006518d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:31.963497 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451628288}
I20260709 13:04:31.994349  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:31.994977  9121 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:31.997792 11987 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:31.998562 11987 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:31.999599  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 854) took 1580 ms. Trace:
I20260709 13:04:32.000181  9859 rpcz_store.cc:276] 0709 13:04:30.418605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.419296 (+   691us) service_pool.cc:224] Handling call
0709 13:04:30.437106 (+ 17810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.743599 (+306493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:31.986800 (+1243201us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54005ee554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:31.999247 (+ 12447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703008128}
I20260709 13:04:31.997861  9115 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:31.998386 11931 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:31.985123 11883 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.011516  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:32.012319  9122 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:32.017190  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.022243  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.022709  9825 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:32.023034  9859 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:32.024601 11986 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:32.024994  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:31.976332 11640 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.041901  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.042294 11640 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:32.042879  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.043238  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.044229  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.044423  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:32.045094  9120 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:31.998322 11974 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.031687 11931 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:32.032971  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.034065 11800 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:32.037060 11976 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:32.038285  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:32.039312  9192 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:32.045404  9114 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:32.047902  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 140210us
I20260709 13:04:32.052790  9122 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:31.982898 11769 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.029838  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.052364  9123 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:31.993194 11970 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:32.053164  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 144291us
I20260709 13:04:32.053706 11883 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:32.041880  9121 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:32.063021 11974 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.065109  9116 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:32.065165  9197 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:32.047570 11923 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.065460  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.065754 11800 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.067353 11769 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:32.068095  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.068508  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:32.069368 11931 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:32.069445  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 44544.5us
I20260709 13:04:32.070782  9856 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:32.052172  9115 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:32.077087  9857 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:32.078007  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:32.078543  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.082947  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.083057  9122 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:32.081010  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:31.999922 11987 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.094604  9121 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.095775  9123 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:32.097748  9825 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:04:32.098708  9087 leader_election.cc:336] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.353s (SENT)
I20260709 13:04:32.100034  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:32.101006  9114 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:32.103263  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.104327  9115 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:32.105187  9087 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:32.102073  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:32.076448 11980 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:32.109586  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.107972  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:32.105249  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:32.110572  9116 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:32.116582  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:32.095001 11586 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEARNER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:32.121498 11970 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:32.059218 11985 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:04:32.122489 11986 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:32.122885 11986 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:32.122970  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:32.109696  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:32.123896 11986 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.124475  9120 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:32.124533  9161 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.125279 11986 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:32.122912  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:32.128208 11836 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:32.128737 11836 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:32.129765 11836 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.131018 11980 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:32.131541 11985 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:32.132014 11872 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:32.132265 11985 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:32.132517 11976 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:32.133484 11980 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:32.118068  9114 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:32.135071  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.123919  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:32.136785  9192 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:32.123020 11769 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:32.137877 11769 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.138288 11769 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.140136  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:32.140264  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 55938.3us
I20260709 13:04:32.140868  9855 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEARNER]: Advancing to term 7
I20260709 13:04:32.143517  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:32.143702  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.144585  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.145510  9861 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:32.118283  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:32.131385  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:32.110374 11969 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:32.083839 11800 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:32.119585 11992 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:32.118902  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:32.147711  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 756331us
I20260709 13:04:32.174045  9121 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:04:32.152926  9198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1878) took 2701 ms (client timeout 1545 ms). Trace:
I20260709 13:04:32.156243  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.127413  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.161734  9857 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:32.152293  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2071) took 1945 ms. Trace:
W20260709 13:04:32.175462  9198 rpcz_store.cc:269] 0709 13:04:29.450874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.466131 (+ 15257us) service_pool.cc:224] Handling call
0709 13:04:32.152490 (+2686359us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400399254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:32.152720 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3203430144}
I20260709 13:04:32.161764  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:32.152164  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 83806.2us
I20260709 13:04:32.167586  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.153077  9855 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEARNER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:32.164641  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:32.177577  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:32.183136 11867 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:32.183547  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:32.183840  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 78914.7us
I20260709 13:04:32.184885  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:32.185241  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:32.185612  9854 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:32.186492  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.186585  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:32.168371  9859 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:32.198263  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 70970.6us
I20260709 13:04:32.164649  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 593) took 1557 ms. Trace:
I20260709 13:04:32.205531  9862 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:32.208323  9114 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:32.210608  9861 raft_consensus.cc:2410] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:32.151314 11977 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:32.153589 11923 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:32.165997 11915 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:32.163656 11791 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:32.164196 11800 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:32.219810  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.215292  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:32.215583  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.160564 11883 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:32.220875  9235 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.199069  9264 rpcz_store.cc:276] 0709 13:04:30.607358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.607462 (+   104us) service_pool.cc:224] Handling call
0709 13:04:30.651062 (+ 43600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.366814 (+715752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:31.366860 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.366909 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:31.366973 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.164281 (+797308us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b5400302fd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:32.164325 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.164478 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6157,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":44889,"spinlock_wait_cycles":1644851968}
I20260709 13:04:32.215245  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:32.224956  9120 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.225905  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.205982 11980 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:32.229156  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:32.177348  9266 rpcz_store.cc:276] 0709 13:04:30.207182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:30.208771 (+  1589us) service_pool.cc:224] Handling call
0709 13:04:30.209045 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:30.297429 (+ 88384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:30.297499 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:30.316181 (+ 18682us) log.cc:844] Serialized 26 byte log entry
0709 13:04:30.595344 (+279163us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:30.948980 (+353636us) log.cc:844] Serialized 14 byte log entry
0709 13:04:30.950348 (+  1368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.151585 (+1201237us) spinlock_profiling.cc:243] Waited 61.3 ms on lock 0x7b5400303ed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:32.151666 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:32.151768 (+   102us) raft_consensus.cc:1685] finished
0709 13:04:32.151877 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.152058 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22763,"prepare.queue_time_us":236503,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":23607,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":19863,"spinlock_wait_cycles":141022080,"thread_start_us":433889,"threads_started":2,"wal-append.queue_time_us":336563,"wal-append.run_cpu_time_us":2231,"wal-append.run_wall_time_us":1254890}
I20260709 13:04:32.188583  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.233281  9121 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:32.215214 11931 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:32.238811  9855 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:32.215469 11977 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:32.215108 11773 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:32.217842 11970 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:32.226943 11706 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:32.246342  9854 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
W20260709 13:04:32.278998  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.389139  9857 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:32.245906 11712 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:32.245869 11790 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:04:32.399861  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.250238 11706 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.247087 11931 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:04:32.402956  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 46387.8us
I20260709 13:04:32.405848  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.412988  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.409152 11706 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:32.241061 11791 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.158s
W20260709 13:04:32.420826  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 193306us
I20260709 13:04:32.243896 11801 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.157s
I20260709 13:04:32.381058 11640 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:32.439879  9857 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:32.398087 11980 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:32.400249 11712 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.460515  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:32.403841 11790 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:32.420149 11773 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:32.461983 11754 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:32.469221  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:04:32.492318  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.494031  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.463817 11712 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:32.507244  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.509029  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2022) took 3041 ms. Trace:
I20260709 13:04:32.509410  9117 rpcz_store.cc:276] 0709 13:04:29.467586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:29.728975 (+261389us) service_pool.cc:224] Handling call
0709 13:04:31.569294 (+1840319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.687189 (+117895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.508777 (+821588us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54002d32d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:32.508946 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4154628352}
I20260709 13:04:32.525915  9857 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:32.474936 11773 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:32.498030  9826 connection.cc:256] RPC call timeout handler was delayed by -1.132069835999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25070000000000003s for a response
W20260709 13:04:32.534665  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 132075us
W20260709 13:04:32.537946  9826 connection.cc:256] RPC call timeout handler was delayed by -1.0028043759999719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
W20260709 13:04:32.538605  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 71855.8us
I20260709 13:04:32.545524  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.499315 11754 raft_consensus.cc:697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:32.542990 11773 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:32.523023 11801 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.559047  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.560189  9862 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:04:32.546710 11967 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:32.486946 11931 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.570516  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1901) took 1540 ms. Trace:
I20260709 13:04:32.583966  9194 rpcz_store.cc:276] 0709 13:04:31.030135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.282864 (+252729us) service_pool.cc:224] Handling call
0709 13:04:31.283480 (+   616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.345090 (+ 61610us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:31.345662 (+   572us) log.cc:844] Serialized 14 byte log entry
0709 13:04:31.418424 (+ 72762us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:31.418500 (+    76us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:31.418597 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.570284 (+1151687us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b54001d5e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:32.570327 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:32.570464 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":392,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":18481,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":50475,"spinlock_wait_cycles":109460736,"thread_start_us":59209,"threads_started":1,"wal-append.queue_time_us":385770}
I20260709 13:04:32.588629 11974 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:32.568406 11836 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:32.589715 11836 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:32.590631 11836 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.592798 11706 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:32.596519 11944 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:32.597084 11944 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:32.598186 11944 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.528369 11980 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:32.573542 11801 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:32.574553 11806 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:32.580602  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:32.567924 11874 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:32.583303  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:32.610432  9855 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:32.609400  9089 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.616854  9120 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:32.622606  9235 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.505463 11977 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.567169 11967 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:32.595625 11706 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:32.628089 11858 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:32.634111 11858 raft_consensus.cc:697] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:32.620518 11874 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:32.639895  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 69508.2us
W20260709 13:04:32.640722  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 60221.5us
I20260709 13:04:32.630329  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:32.616246 11806 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.636210  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:32.664248  9271 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:32.635066 11858 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.639767 11967 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:32.637743 11517 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:32.647204 11874 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:32.643874 11926 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.679158 11773 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.640249 11727 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:32.680791  9265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 607) took 2063 ms (client timeout 2214 ms). Trace:
W20260709 13:04:32.689570  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 67053.2us
I20260709 13:04:32.680907 11754 consensus_queue.cc:237] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:32.684399  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 54201.6us
W20260709 13:04:32.694458  9265 rpcz_store.cc:269] 0709 13:04:30.617335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.071615 (+454280us) service_pool.cc:224] Handling call
0709 13:04:32.680554 (+1608939us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54000eb0d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:32.680717 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1611952384}
I20260709 13:04:32.696552  9114 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:32.666038 11806 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:32.698092  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 141003us
I20260709 13:04:32.696707 11791 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:32.645170 11825 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:04:32.689544 11977 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:32.728688  9826 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.748s (SENT)
I20260709 13:04:32.726511  9269 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:04:32.699617 11727 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:32.731442 11727 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:32.731873 12005 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.067s	user 0.000s	sys 0.003s
I20260709 13:04:32.734643 11980 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:32.735375  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:32.735280 11980 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:32.736162  9114 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:32.737269  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.737505  9235 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.640453 11790 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:32.739058 11790 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:32.740976 11773 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:32.741806  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:32.742574  9114 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:32.744628 11969 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:32.729892  9269 pending_rounds.cc:85] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 6
W20260709 13:04:32.728185 11769 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.000s
I20260709 13:04:32.751655  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:32.698323 11800 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:04:32.749198  9269 pending_rounds.cc:107] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:04:32.759649  9269 raft_consensus.cc:2889] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:32.741751 11773 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:32.710248 11926 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.742574 11967 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.775975 11967 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:32.784574 11725 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:32.785036 11725 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:32.785965 11725 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.760680 11640 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:32.730167 11883 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:04:32.763136 11773 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:32.816125  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:32.759092 11769 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.794900 11640 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.809863 11773 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:32.864281  9859 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:04:32.848304 11640 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:32.875926  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 206374us
W20260709 13:04:32.886960  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 161148us
I20260709 13:04:32.871327  9271 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:32.853973 11801 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:32.901175  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.902007  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:32.908938  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 149549us
I20260709 13:04:32.902457  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:32.898487 11994 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.348s	user 0.001s	sys 0.002s
I20260709 13:04:32.892414 11969 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:32.891762 11915 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.919384 11883 raft_consensus.cc:3055] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:32.920804 11883 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:32.919458 11915 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.922415 11883 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:32.924502 11883 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:32.925104 11969 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.929076 11915 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:32.934208  9860 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 2.
I20260709 13:04:32.894976 11974 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:32.937412  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 163811us
I20260709 13:04:32.943141  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:32.943809  9266 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:32.944818  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.890992 11712 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:32.932134 11800 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:04:32.946770  9826 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.976s (SENT)
W20260709 13:04:32.951885  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:32.968101  9122 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:32.972671  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:32.972332  9826 leader_election.cc:336] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.994s (SENT)
I20260709 13:04:32.975191  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:32.940487 11915 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:32.978399 11915 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:32.945256 11975 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:32.983394  9264 raft_consensus.cc:2393] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:32.939096 11791 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:32.993809  9122 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:33.001849  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:33.006593  9117 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:33.005692  9120 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:33.009131  9121 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:32.947175 12016 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:32.954344 11706 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.962536 11793 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:33.037617 12029 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.038229 12029 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:33.038921 11806 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.040023 11791 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.040720 12029 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:33.040048 11915 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.046187  9271 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:33.048084  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 78927.6us
I20260709 13:04:33.035475  9114 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:32.985059 11801 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:04:32.985653 11975 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:33.009084 12026 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.023375 12018 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:33.013144 11679 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:33.056164  9235 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:33.043471 12016 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:32.951292 11974 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:32.967867 11800 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.043295 11806 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:33.064260  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:32.972126 12027 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 13:04:33.067013  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 94728.3us
I20260709 13:04:33.055277 11769 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:33.048013 11926 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:33.085937  9826 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.214s (SENT)
I20260709 13:04:33.086395  9117 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:33.086453  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:33.080343 12016 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:33.087998 11769 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:33.048573 11727 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:33.090955 11727 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:33.091223  9826 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.801s (SENT)
I20260709 13:04:33.091816  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:33.092540 11769 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:33.061856 12018 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:33.100646 12018 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:33.102034 12018 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:33.108675  9117 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:33.108836  9119 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:33.111356 11790 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:33.121990  9266 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:33.055095 11706 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:33.129565  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45436.1us
W20260709 13:04:33.132171  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.069947 12028 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:33.058369 11793 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.141693  9194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:33.143128  9194 raft_consensus.cc:606] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:33.146958  9194 raft_consensus.cc:614] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:33.148840  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.151427  9191 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:33.150547  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:33.138576 12028 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.067247 12026 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:33.154130  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.154434  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:33.155099  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:33.152595  9268 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:33.157140  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:33.160650  9270 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:33.159500 11926 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:04:33.162393  9197 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:33.168215 11706 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:33.168669 11712 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:33.143668 11975 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:33.169512 12019 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:04:33.166950  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 1966) took 1105 ms. Trace:
I20260709 13:04:33.170863  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:33.172714  9826 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.828s (SENT)
I20260709 13:04:33.172868  9123 rpcz_store.cc:276] 0709 13:04:32.061138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.067793 (+  6655us) service_pool.cc:224] Handling call
0709 13:04:33.166786 (+1098993us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b54001e3f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:33.166893 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36552448}
I20260709 13:04:33.173310  9826 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:33.176105  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.176651  9194 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:33.180516  9164 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:33.159448 12023 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:04:33.165546 12017 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:04:33.156713 12028 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:33.205745  9268 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:33.206111 11975 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:33.183163  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 69247us
I20260709 13:04:33.210059  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:33.211027  9122 raft_consensus.cc:2393] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:33.211417 11995 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.812s	user 0.002s	sys 0.000s
I20260709 13:04:33.215958 12023 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:33.165586 11769 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:33.216538 11769 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.216971 11769 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:33.185622  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.218020  9197 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:33.172725 11728 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:33.186184  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:04:33.226405  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45936.7us
I20260709 13:04:33.227458  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:33.186550 11679 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:33.186798 12040 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:33.184201  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:33.187646  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:33.188058 11800 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:33.189591  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:33.190285  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:33.191358  9269 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:33.192055  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:33.192185  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:33.193290  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:33.193559 11874 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.194636  9272 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:33.198051 12026 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:33.199856 11793 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:33.200978 11974 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:33.215541  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:33.222865  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.321874  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 73099.3us
I20260709 13:04:33.350991 11974 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:33.320449  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:33.351903 12027 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:33.321985 12032 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:04:33.352641  9858 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:33.353613 12039 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:33.353960  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:33.354032 12039 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:33.293457 11773 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:33.301357 11806 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:33.359787  9266 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:33.321735  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:33.355270 12039 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:33.356267 12004 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.683s	user 0.005s	sys 0.000s
I20260709 13:04:33.355306 11679 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.351399  9265 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:33.351760 11974 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.379612  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 829) took 1214 ms. Trace:
I20260709 13:04:33.380055  9853 rpcz_store.cc:276] 0709 13:04:32.164902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.518675 (+353773us) service_pool.cc:224] Handling call
0709 13:04:32.519135 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:32.519689 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.519715 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.520350 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.649815 (+129465us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:32.892920 (+243105us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.893220 (+   300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.893316 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.376029 (+482713us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b180040d080. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:33.379325 (+  3296us) raft_consensus.cc:1685] finished
0709 13:04:33.379428 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.379551 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70833,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":1659,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":7369,"spinlock_wait_cycles":4851584,"thread_start_us":318597,"threads_started":2,"wal-append.queue_time_us":226663}
I20260709 13:04:33.382241  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:33.382962 11769 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.217033 12023 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.383503 11679 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.358152  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:33.384650  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.384709  9859 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
W20260709 13:04:33.385115  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:33.277354  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49953.5us
I20260709 13:04:33.292721 11969 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.388772 11969 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.388828  9164 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:33.364248 11801 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:33.391680 11817 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:33.359962 11980 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.392177 11817 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:33.320838  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:33.392344  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:33.392825  9857 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:33.392519 11980 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:33.393123  9265 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:33.394583 11980 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:04:33.396096  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:33.292774 11728 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:33.398306  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:33.358635  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 143161us
I20260709 13:04:33.362159  9119 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:33.402058  9857 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:33.361585  9120 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:33.228536  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:33.321125  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:33.321516  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:33.384076 11970 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.391435 11801 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.357127 11874 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:33.405236  9270 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:33.253599  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 80920.1us
I20260709 13:04:33.400676  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 815) took 1242 ms. Trace:
I20260709 13:04:33.409139  9263 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:33.406499 11817 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:33.408569 11728 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:04:33.413529  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:33.454289  9122 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:33.414660  9121 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:33.408797  9861 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
W20260709 13:04:33.415575  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 230727us
I20260709 13:04:33.416009  9266 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39750
I20260709 13:04:33.404353  9855 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:33.420009  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:33.455492  9266 raft_consensus.cc:606] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Received request to transfer leadership
I20260709 13:04:33.419948 11872 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:33.420348 11970 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.421348  9238 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:33.421624 11800 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:33.421725 11944 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:33.359966 11825 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:33.426206 11773 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:33.427069 12039 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:33.429461  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:33.430030  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:33.432693 11915 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:33.434516 11801 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:33.444264  9862 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:04:33.487883  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.488300  9862 pending_rounds.cc:85] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 5
I20260709 13:04:33.488646  9862 pending_rounds.cc:107] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:04:33.488951  9862 raft_consensus.cc:2889] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:33.445595 11517 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:33.278119 11977 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.439356  9265 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:33.249063 11712 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:33.455847  9266 raft_consensus.cc:614] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:33.455858  9120 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:33.466123  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:33.467986  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:33.468181  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.465279  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:33.205726 11640 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.474431 11800 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:33.481120  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.483647  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:33.484509  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:33.450711  9856 rpcz_store.cc:276] 0709 13:04:32.158698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.158828 (+   130us) service_pool.cc:224] Handling call
0709 13:04:32.159154 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:32.159582 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:32.159608 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:32.160149 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:04:32.243104 (+ 82955us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:32.243159 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:32.244679 (+  1520us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:33.204264 (+959585us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:33.400379 (+196115us) raft_consensus.cc:1685] finished
0709 13:04:33.400398 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.400618 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1842,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":32432512,"thread_start_us":82565,"threads_started":1,"wal-append.queue_time_us":470622}
I20260709 13:04:33.486788  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:33.485364  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:33.483767  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.492537  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1950) took 1571 ms. Trace:
I20260709 13:04:33.493191  9194 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:04:33.474133 11969 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:33.480968  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:33.498594  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 77348.5us
I20260709 13:04:33.496290  9860 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:33.500422  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:33.504312  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.465430 11926 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:04:33.505301  9190 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:04:33.880879  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 165392us
I20260709 13:04:33.959046  9268 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:33.505357  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:33.508864  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:33.505668  9197 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:33.505718  9861 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:33.961611  9854 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:33.663949  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:33.987892  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:33.782397  9191 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:33.664839  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.992419  9118 rpcz_store.cc:276] 0709 13:04:31.921436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:31.921700 (+   264us) service_pool.cc:224] Handling call
0709 13:04:31.943529 (+ 21829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:31.978378 (+ 34849us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:32.222100 (+243722us) log.cc:844] Serialized 14 byte log entry
0709 13:04:32.420420 (+198320us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:32.420472 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:32.420519 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:33.076297 (+655778us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b18003b89c0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535c52a6 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419
0709 13:04:33.492382 (+416085us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b540034e754. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:33.492421 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:33.492496 (+    75us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70420,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":134,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":24667,"spinlock_wait_cycles":92263552,"thread_start_us":389359,"threads_started":2,"wal-append.queue_time_us":378628,"wal-append.run_cpu_time_us":1345,"wal-append.run_wall_time_us":1026295}
W20260709 13:04:33.494727 11993 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 1.091s	user 0.000s	sys 0.004s
W20260709 13:04:34.004629  9122 consensus_meta.cc:287] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 13:04:34.005303  9122 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:33.790162  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:33.989836  9198 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:33.494091 11825 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:33.493844 12035 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.603s
W20260709 13:04:34.020273  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:34.604875  9114 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.165s	user 0.052s	sys 0.091s
I20260709 13:04:34.173238 12053 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:34.616391  9114 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.176s	user 0.053s	sys 0.092s
I20260709 13:04:34.013741 12023 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.617400  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2061) took 1409 ms. Trace:
I20260709 13:04:33.679867 11980 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:34.617646 12023 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.124518  9854 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:34.617944  9114 rpcz_store.cc:276] 0709 13:04:33.208297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.208421 (+   124us) service_pool.cc:224] Handling call
0709 13:04:33.208736 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:33.209105 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:33.209132 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:33.209563 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:04:33.440130 (+230567us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b1800194620. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:04:34.616947 (+1176817us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:34.616996 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.617078 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:34.617116 (+    38us) raft_consensus.cc:1685] finished
0709 13:04:34.617126 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.617341 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":604,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":10923264,"thread_start_us":1176581,"threads_started":1,"wal-append.queue_time_us":1404721}
I20260709 13:04:34.619340  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 834) took 1283 ms. Trace:
W20260709 13:04:34.179960 11944 consensus_meta.cc:287] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.000s
I20260709 13:04:34.619905  9854 rpcz_store.cc:276] 0709 13:04:33.336227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.465066 (+128839us) service_pool.cc:224] Handling call
0709 13:04:34.619260 (+1154194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.621011  9271 raft_consensus.cc:2410] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:04:34.621583  9268 consensus_meta.cc:287] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.607s	user 0.002s	sys 0.030s
I20260709 13:04:34.622014  9268 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
W20260709 13:04:34.622576  9268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 617) took 1204 ms (client timeout 1548 ms). Trace:
I20260709 13:04:34.614324 11725 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:34.219141 11970 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:34.098754  9861 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 3.
I20260709 13:04:34.624752  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 832) took 1291 ms. Trace:
I20260709 13:04:34.625168  9861 rpcz_store.cc:276] 0709 13:04:33.333243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.464887 (+131644us) service_pool.cc:224] Handling call
0709 13:04:34.624674 (+1159787us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:33.508096 11712 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:34.626288 11915 consensus_meta.cc:287] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.135s	user 0.002s	sys 0.007s
I20260709 13:04:34.024330  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:33.505573 11640 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:34.628345 11640 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.615530  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 1.12881e+06us
I20260709 13:04:34.034099 11969 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:33.987054 11800 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:34.579948 11817 consensus_meta.cc:287] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.084s	user 0.003s	sys 0.006s
I20260709 13:04:34.633481 11817 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:34.040298 12048 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:04:33.779376 11801 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:33.753844 12050 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 13:04:34.629086 11640 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:33.688469 11982 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.074574  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.038842  9265 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
W20260709 13:04:34.644466  9826 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.995s (SENT) [suppressed 48 similar messages]
W20260709 13:04:34.666772  9265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1698) took 1254 ms (client timeout 1658 ms). Trace:
W20260709 13:04:34.667192  9265 rpcz_store.cc:269] 0709 13:04:33.412751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.500148 (+ 87397us) service_pool.cc:224] Handling call
0709 13:04:34.666694 (+1166546us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:34.026486 12055 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:04:34.105593  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.669602  9267 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:34.024185  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:34.616797 12053 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.635221 11915 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:34.637805  9195 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
W20260709 13:04:34.510160  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 720035us
W20260709 13:04:34.195205  9119 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.786s	user 0.007s	sys 0.001s
I20260709 13:04:34.676276  9119 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:34.316511 11728 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:34.678488  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.144376 12018 raft_consensus.cc:1064] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
W20260709 13:04:34.636808 12039 consensus_meta.cc:287] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.149s	user 0.007s	sys 0.004s
I20260709 13:04:34.672504  9192 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:34.670192  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1690) took 1274 ms (client timeout 1683 ms). Trace:
W20260709 13:04:34.657394 12032 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.006s	sys 0.007s
I20260709 13:04:34.300707 12026 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.676600  9196 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 13:04:34.676957  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 1974) took 1502 ms. Trace:
W20260709 13:04:34.236397  9270 consensus_meta.cc:287] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.784s	user 0.005s	sys 0.005s
W20260709 13:04:34.664106  9236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.004s	sys 0.007s
W20260709 13:04:34.686980  9236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.005s	sys 0.007s
W20260709 13:04:34.687901  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:34.688457  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:34.622960  9268 rpcz_store.cc:269] 0709 13:04:33.418061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.478954 (+ 60893us) service_pool.cc:224] Handling call
0709 13:04:34.622520 (+1143566us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.690335  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:34.691399  9268 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:34.655133 11970 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:34.686069  9270 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:33.498894 11977 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:34.692870  9825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.005s	sys 0.010s
W20260709 13:04:34.697249  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 66098us
W20260709 13:04:34.697634  9825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.005s	sys 0.010s
I20260709 13:04:34.685823  9119 rpcz_store.cc:276] 0709 13:04:33.175005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.175128 (+   123us) service_pool.cc:224] Handling call
0709 13:04:34.676888 (+1501760us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:34.697434  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 673327us
I20260709 13:04:34.698518  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:34.682969  9267 rpcz_store.cc:269] 0709 13:04:33.396220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.015565 (+619345us) service_pool.cc:224] Handling call
0709 13:04:34.670138 (+654573us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:34.685781  9235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.011s	sys 0.003s
W20260709 13:04:34.700493  9235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.011s	sys 0.003s
I20260709 13:04:34.703361  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:34.704674  9861 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:04:34.703560  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:34.699610  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:04:34.686875  9272 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.015s	sys 0.034s
W20260709 13:04:34.707402  9272 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.015s	sys 0.035s
I20260709 13:04:34.665620 12023 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:34.706972  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 480) took 1003 ms. Trace:
I20260709 13:04:33.678900 12028 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:34.708359  9196 rpcz_store.cc:276] 0709 13:04:33.682928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.654648 (+971720us) service_pool.cc:224] Handling call
0709 13:04:34.676314 (+ 21666us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:34.685688 (+  9374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.685950 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:34.712690  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.713249  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1687) took 1318 ms. Trace:
I20260709 13:04:34.708824  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:34.715600  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:34.716428  9853 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:34.685212 12032 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.897s	user 0.006s	sys 0.008s
I20260709 13:04:34.718930  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:34.720094  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:34.720970  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:34.717139  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:34.724761  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.698658 11773 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:34.724576  9856 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:34.726593  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:34.727209  9854 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:34.720638  9860 raft_consensus.cc:2376] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEARNER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7. Re-sending same reply.
I20260709 13:04:34.728782  9265 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:34.699100 11791 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:34.713703  9272 rpcz_store.cc:276] 0709 13:04:33.395152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.409590 (+ 14438us) service_pool.cc:224] Handling call
0709 13:04:33.469847 (+ 60257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.470045 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:33.499065 (+ 29020us) log.cc:844] Serialized 14 byte log entry
0709 13:04:34.712844 (+1213779us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:34.712902 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:34.712996 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.713071 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:34.713196 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4874,"prepare.queue_time_us":19205,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"thread_start_us":910009,"threads_started":1}
I20260709 13:04:34.714771 11969 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:34.731119  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54905.4us
W20260709 13:04:34.714232 11975 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.003s	sys 0.013s
I20260709 13:04:34.732008  9858 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:34.699646 11728 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:34.710924 11883 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:34.743928 12065 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:34.744130 11883 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:34.744709 11883 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:34.744626 12065 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.674000 11801 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.718247 12039 raft_consensus.cc:2764] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:04:34.729489 12069 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.734470  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:34.730444  9268 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
W20260709 13:04:34.736334  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 620) took 1311 ms (client timeout 1729 ms). Trace:
W20260709 13:04:34.752719  9267 rpcz_store.cc:269] 0709 13:04:33.419026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.700388 (+1281362us) service_pool.cc:224] Handling call
0709 13:04:34.730690 (+ 30302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.755344 12069 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.755993 11640 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:34.756842 12053 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:34.734409 12064 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:34.757876 12064 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.764923 11817 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:34.753742  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:34.734474  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:34.714006 12023 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:34.671201 11982 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.737671  9853 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:34.718824 11712 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:34.770015 12023 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.747278  9860 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:34.716352 11970 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:34.768532  9268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 618) took 1350 ms (client timeout 1717 ms). Trace:
I20260709 13:04:34.772097 12023 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:34.772380  9268 rpcz_store.cc:269] 0709 13:04:33.418306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.689972 (+1271666us) service_pool.cc:224] Handling call
0709 13:04:34.768439 (+ 78467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.774641 11974 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:34.775210  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:34.749234  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:34.712221 12026 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:34.750340 12067 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.770458 11801 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:34.778456  9268 raft_consensus.cc:2376] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5. Re-sending same reply.
I20260709 13:04:34.786022  9269 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:34.778019  9272 raft_consensus.cc:2376] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7. Re-sending same reply.
W20260709 13:04:34.797971  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 621) took 1378 ms (client timeout 1729 ms). Trace:
W20260709 13:04:34.797663  9268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1707) took 1382 ms (client timeout 1799 ms). Trace:
W20260709 13:04:34.789144  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1709) took 1372 ms (client timeout 1774 ms). Trace:
W20260709 13:04:34.791172  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 56836us
I20260709 13:04:34.817744 11754 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:34.818545 11754 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:34.798473  9272 rpcz_store.cc:269] 0709 13:04:33.419474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.734196 (+1314722us) service_pool.cc:224] Handling call
0709 13:04:34.797912 (+ 63716us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:34.828948  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:34.828934  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:34.829312  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 90643.5us
I20260709 13:04:34.796135 12070 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:04:34.830648  9854 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:34.830641  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:34.832885  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:34.830456  9826 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.693s (SENT)
W20260709 13:04:34.799540  9268 rpcz_store.cc:269] 0709 13:04:33.415397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.774926 (+1359529us) service_pool.cc:224] Handling call
0709 13:04:34.797581 (+ 22655us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:34.801465  9267 rpcz_store.cc:269] 0709 13:04:33.416659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.753456 (+1336797us) service_pool.cc:224] Handling call
0709 13:04:34.789063 (+ 35607us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.839834  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:34.839936  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:34.788393 12053 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:34.841184  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:34.842990  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:34.827869 11980 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.851078  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.748457 11791 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:34.855458  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:34.854287  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:34.858515  9119 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:34.768005 12071 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 13:04:34.747818 11825 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:34.831166 11974 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.797262 11800 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.864563  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.839068 11640 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:34.854952  9272 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:34.867771  9114 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:34.849329 12035 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:34.875445  9118 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:34.869496  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:34.870570  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.876621  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:34.863610 11725 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.872440  9267 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
W20260709 13:04:34.877700  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1689) took 1481 ms (client timeout 1937 ms). Trace:
W20260709 13:04:34.878111  9267 rpcz_store.cc:269] 0709 13:04:33.396003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.849218 (+1453215us) service_pool.cc:224] Handling call
0709 13:04:34.877638 (+ 28420us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.879084  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.788211 12067 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:34.874523  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 64364.8us
I20260709 13:04:34.880918  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2087) took 2008 ms. Trace:
W20260709 13:04:34.881476  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1708) took 1465 ms (client timeout 1821 ms). Trace:
I20260709 13:04:34.881311  9263 rpcz_store.cc:276] 0709 13:04:32.872197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:32.872373 (+   176us) service_pool.cc:224] Handling call
0709 13:04:32.890861 (+ 18488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.453579 (+562718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:34.880708 (+1427129us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b5400061b54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:34.880865 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191558016}
I20260709 13:04:34.882673  9855 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:34.870272 12023 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.882242  9272 rpcz_store.cc:269] 0709 13:04:33.415644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.829035 (+1413391us) service_pool.cc:224] Handling call
0709 13:04:34.881417 (+ 52382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.856237 12065 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:34.881261  9826 leader_election.cc:336] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.729s (SENT)
I20260709 13:04:34.852629 12053 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.886121 11712 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:34.884877  9268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1694) took 1487 ms (client timeout 1928 ms). Trace:
W20260709 13:04:34.902179  9268 rpcz_store.cc:269] 0709 13:04:33.397120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.864264 (+1467144us) service_pool.cc:224] Handling call
0709 13:04:34.884820 (+ 20556us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:34.887140  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 625) took 1461 ms (client timeout 1914 ms). Trace:
I20260709 13:04:34.890805 12073 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:04:34.903204  9267 rpcz_store.cc:269] 0709 13:04:33.420520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.878827 (+1458307us) service_pool.cc:224] Handling call
0709 13:04:34.881693 (+  2866us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.855487 11980 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.887284 11679 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:34.880440 11640 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.892459 11769 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:34.875308 12035 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:34.917634 12035 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:34.919270 11640 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:34.895321 11800 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:34.895861 12023 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.896767 11791 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:34.873591 11974 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.856128 11969 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.943058  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:34.898813  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:34.877560 11975 raft_consensus.cc:670] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:04:34.900635  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:34.964865  9854 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:34.891273  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.965591  9263 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
W20260709 13:04:34.966081  9263 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 622) took 1546 ms (client timeout 1942 ms). Trace:
W20260709 13:04:34.966507  9263 rpcz_store.cc:269] 0709 13:04:33.419785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.891001 (+1471216us) service_pool.cc:224] Handling call
0709 13:04:34.966013 (+ 75012us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.905309  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:34.905622  9122 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
I20260709 13:04:34.906579  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.968075  9268 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
W20260709 13:04:34.968786  9268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 623) took 1548 ms (client timeout 1942 ms). Trace:
W20260709 13:04:34.969292  9268 rpcz_store.cc:269] 0709 13:04:33.420035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.902882 (+1482847us) service_pool.cc:224] Handling call
0709 13:04:34.968708 (+ 65826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.883713  9119 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:34.912776 11754 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:34.851851 11728 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:34.915119  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:34.973968  9267 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:34.883190 11801 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:34.903700 11970 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:34.893334 12061 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:04:34.920382 12065 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:34.874161 12075 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:34.907886  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:34.981184 12026 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:34.981665  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2103) took 1532 ms (client timeout 1888 ms). Trace:
I20260709 13:04:34.981824 12074 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:34.982087  9272 rpcz_store.cc:269] 0709 13:04:33.449437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.885783 (+1436346us) service_pool.cc:224] Handling call
0709 13:04:34.981603 (+ 95820us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:34.927139  9114 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:34.897117 12064 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:34.887590 12076 raft_consensus.cc:2764] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:04:34.924225 11679 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:34.931990 12023 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:34.932578  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 56043us
I20260709 13:04:34.939366  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:34.929034 11769 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.011s
W20260709 13:04:34.939904  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 58672.4us
W20260709 13:04:34.940893  9265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1702) took 1526 ms (client timeout 1710 ms). Trace:
I20260709 13:04:34.941613  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:34.942291  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:34.947775  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:34.902938 12067 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:34.935250 12035 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:34.887279 11725 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:34.927423 12080 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:34.927416 12028 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:34.982355 12074 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.135836  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2096) took 1947 ms. Trace:
I20260709 13:04:35.140476  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:35.144338  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:35.143054  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.140689  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:35.150715  9826 leader_election.cc:336] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.729s (SENT)
W20260709 13:04:35.132447  9192 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1908) took 1663 ms (client timeout 1915 ms). Trace:
I20260709 13:04:35.092273  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.153002  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.092626  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.151564  9271 rpcz_store.cc:276] 0709 13:04:33.188861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.191864 (+  3003us) service_pool.cc:224] Handling call
0709 13:04:35.135616 (+1943752us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54000faad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:35.135774 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2903448960}
I20260709 13:04:35.152057  9857 raft_consensus.cc:2393] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:35.154428  9263 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:35.037484  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.154825  9238 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:35.155072  9263 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1692) took 1758 ms (client timeout 2066 ms). Trace:
W20260709 13:04:35.095191  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.132182  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.155920  9121 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:35.156193  9272 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:35.130836  9854 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:35.156747  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1696) took 1759 ms (client timeout 2196 ms). Trace:
W20260709 13:04:35.157087  9272 rpcz_store.cc:269] 0709 13:04:33.397591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.118855 (+1721264us) service_pool.cc:224] Handling call
0709 13:04:35.156698 (+ 37843us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.140244  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.118014  9267 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
W20260709 13:04:35.158088  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 619) took 1739 ms (client timeout 1995 ms). Trace:
I20260709 13:04:35.155215  9268 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:35.048352  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:35.117089  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 252948us
W20260709 13:04:35.150089  9265 rpcz_store.cc:269] 0709 13:04:33.414057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.678229 (+1264172us) service_pool.cc:224] Handling call
0709 13:04:34.940530 (+262301us) spinlock_profiling.cc:243] Waited 49.4 ms on lock 0x7b540032e454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:34.940814 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113561344}
I20260709 13:04:34.968948 12032 raft_consensus.cc:3055] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:04:35.153297  9192 rpcz_store.cc:269] 0709 13:04:33.468668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.468764 (+    96us) service_pool.cc:224] Handling call
0709 13:04:35.132235 (+1663471us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54004169d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:35.132387 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1643140480}
W20260709 13:04:35.166044  9825 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 2.874s (SENT)
W20260709 13:04:35.119462  9087 connection.cc:256] RPC call timeout handler was delayed by -1.4378219579999723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 13:04:35.184453  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.167832  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.184930  9162 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:35.158499  9267 rpcz_store.cc:269] 0709 13:04:33.418753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.907114 (+1488361us) service_pool.cc:224] Handling call
0709 13:04:35.158033 (+250919us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:35.185431  9089 connection.cc:256] RPC call timeout handler was delayed by -1.7643243910000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
W20260709 13:04:35.183852  9268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1700) took 1770 ms (client timeout 2061 ms). Trace:
W20260709 13:04:35.090152  9161 connection.cc:256] RPC call timeout handler was delayed by -1.5028201599999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 13:04:35.158618  9263 rpcz_store.cc:269] 0709 13:04:33.396676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.967265 (+1570589us) service_pool.cc:224] Handling call
0709 13:04:35.154969 (+187704us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:35.189975  9268 rpcz_store.cc:269] 0709 13:04:33.413416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:34.970248 (+1556832us) service_pool.cc:224] Handling call
0709 13:04:35.183789 (+213541us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.190464  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:35.190842  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.186812  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.189431 12032 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:35.192600  9265 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:35.077518  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:35.115037  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:35.043110  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 121500us
W20260709 13:04:35.184533  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.020871 12068 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.165917  9116 raft_consensus.cc:2410] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:35.122143 12084 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.190411  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:35.128324 11982 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.193436  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:35.185899  9089 connection.cc:256] RPC call timeout handler was delayed by -1.7049243909999632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
W20260709 13:04:35.215801  9089 connection.cc:256] RPC call timeout handler was delayed by -1.6401243909999721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
W20260709 13:04:35.216288  9089 connection.cc:256] RPC call timeout handler was delayed by -1.2954243909999832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
W20260709 13:04:35.216852  9089 connection.cc:256] RPC call timeout handler was delayed by -1.1784243909999645s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 13:04:35.217262  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0242065309999475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
I20260709 13:04:35.219472  9117 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:35.220429  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.188136  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.130120 11926 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:35.195487  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.235366  9860 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:35.195667  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.236491  9268 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:35.237104  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1691) took 1840 ms. Trace:
I20260709 13:04:35.237495  9268 rpcz_store.cc:276] 0709 13:04:33.396460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.190777 (+1794317us) service_pool.cc:224] Handling call
0709 13:04:35.237055 (+ 46278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.186443 12076 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:35.194491  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:35.195382  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.197425  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:35.247984  9857 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:35.196478  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.134011 11974 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:35.199081  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.020874 11883 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.087918 11679 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.200457  9859 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:35.193831 12086 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:35.252259  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.135085 12053 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.191067  9121 raft_consensus.cc:2376] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5. Re-sending same reply.
I20260709 13:04:35.203686  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 627) took 1773 ms. Trace:
I20260709 13:04:35.205467  9119 raft_consensus.cc:2376] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5. Re-sending same reply.
I20260709 13:04:35.193295  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.205965  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.206801  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.266158  9272 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:35.266601  9114 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:35.266809  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1699) took 1853 ms (client timeout 2267 ms). Trace:
W20260709 13:04:35.267372  9272 rpcz_store.cc:269] 0709 13:04:33.413093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.187365 (+1774272us) service_pool.cc:224] Handling call
0709 13:04:35.266737 (+ 79372us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:35.213228  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 58479.7us
I20260709 13:04:35.269191  9265 rpcz_store.cc:276] 0709 13:04:33.420951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.186532 (+1765581us) service_pool.cc:224] Handling call
0709 13:04:35.194661 (+  8129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.234803 11926 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:35.218703  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.208555 11517 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.221748 12026 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:35.224936  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.228224 12079 consensus_queue.cc:1059] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 13:04:35.286015  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 67437.5us
I20260709 13:04:35.289047  9117 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:35.284744  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:35.198573 12067 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:35.250046  9271 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 8 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:04:35.249920  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1704) took 1835 ms. Trace:
I20260709 13:04:35.305708  9263 rpcz_store.cc:276] 0709 13:04:33.414589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.190609 (+1776020us) service_pool.cc:224] Handling call
0709 13:04:35.249863 (+ 59254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.250316  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:35.306317  9271 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 626) took 1885 ms (client timeout 2230 ms). Trace:
I20260709 13:04:35.253674 12032 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.215049 12068 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.191416 11825 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:04:35.187671 12080 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.256778  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.257499  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.258203  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:35.313015  9118 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:35.285318 12078 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:35.264199  9123 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:35.202272  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:35.222726  9267 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:35.313907  9196 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:35.244189 12028 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:35.265588  9853 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:35.280614  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:35.280614  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.317524  9265 raft_consensus.cc:2410] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:35.318039  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 615) took 1900 ms. Trace:
I20260709 13:04:35.318369  9265 rpcz_store.cc:276] 0709 13:04:33.417434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.280261 (+1862827us) service_pool.cc:224] Handling call
0709 13:04:35.317978 (+ 37717us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.319686  9265 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:35.320307  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1697) took 1922 ms. Trace:
I20260709 13:04:35.320634  9265 rpcz_store.cc:276] 0709 13:04:33.403866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.319036 (+1915170us) service_pool.cc:224] Handling call
0709 13:04:35.319357 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.320045 (+   688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.320227 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.285959  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.285364  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:35.194187  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:35.322060  9192 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:35.231986 12084 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:35.290910  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:04:35.324627  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 122462us
W20260709 13:04:35.322386  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 128277us
I20260709 13:04:35.290474  9272 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.259675 12086 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:35.264549 12082 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:35.199980  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49287.9us
W20260709 13:04:35.330025  9826 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.914s (SENT)
I20260709 13:04:35.306555  9268 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:35.330536  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:35.330787  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 614) took 1913 ms. Trace:
I20260709 13:04:35.331115  9268 rpcz_store.cc:276] 0709 13:04:33.417182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.238352 (+1821170us) service_pool.cc:224] Handling call
0709 13:04:35.330729 (+ 92377us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:35.306722  9271 rpcz_store.cc:269] 0709 13:04:33.420727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.186641 (+1765914us) service_pool.cc:224] Handling call
0709 13:04:35.306261 (+119620us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.306903  9263 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:35.291278 12081 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.058s	user 0.001s	sys 0.001s
I20260709 13:04:35.310242  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.287408 11883 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:35.251523 12076 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:35.334851  9197 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:35.332275  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1693) took 1935 ms. Trace:
W20260709 13:04:35.312386  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63071.1us
I20260709 13:04:35.200265 11867 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.052s
W20260709 13:04:35.323380  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1695) took 1926 ms (client timeout 2565 ms). Trace:
I20260709 13:04:35.314661 12032 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.330035  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.332798  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.285073 11517 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.285017 11926 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.360860 11926 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.361464 11517 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:35.362145 11969 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:35.362610 11969 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.362990 11969 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.310302 12026 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:35.365779  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.333606  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:35.335868  9263 rpcz_store.cc:276] 0709 13:04:33.396895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.306442 (+1909547us) service_pool.cc:224] Handling call
0709 13:04:35.306735 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.332032 (+ 25297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.332218 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:35.369306  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.337626  9271 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:35.370316  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 616) took 1952 ms. Trace:
I20260709 13:04:35.370630  9271 rpcz_store.cc:276] 0709 13:04:33.417678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.332789 (+1915111us) service_pool.cc:224] Handling call
0709 13:04:35.337319 (+  4530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.370084 (+ 32765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.370266 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.338120 12035 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.338260 12078 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:35.338640 11725 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:35.340402 12084 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.315934 11975 consensus_queue.cc:1059] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:04:35.340777 12076 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.340615 12082 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.342638 11679 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.328612 12086 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:35.334654 12067 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:35.330600 12071 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:35.329341 11728 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:35.346503  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.347119  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.347805  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.348417  9853 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:04:35.350590  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.424449  9196 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
W20260709 13:04:35.352394  9826 leader_election.cc:336] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.942s (SENT)
I20260709 13:04:35.352846 12053 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:35.428174 11728 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:35.354367  9267 rpcz_store.cc:269] 0709 13:04:33.397349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.187901 (+1790552us) service_pool.cc:224] Handling call
0709 13:04:35.323331 (+135430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.428732 11728 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:35.430044  9266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 624) took 2009 ms (client timeout 1693 ms). Trace:
W20260709 13:04:35.430480  9266 rpcz_store.cc:269] 0709 13:04:33.420260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.429828 (+2009568us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:35.429958 (+   130us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:04:35.336606  9087 connection.cc:256] RPC call timeout handler was delayed by -1.3565770059999522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 13:04:35.432039  9087 connection.cc:256] RPC call timeout handler was delayed by -1.2740646219999689s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
W20260709 13:04:35.432571  9087 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.499s (SENT)
I20260709 13:04:35.432971  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:35.436316 12076 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:35.437510  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:35.437683  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 235184us
I20260709 13:04:35.438321  9198 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:35.370774  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:35.439600  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 68906.4us
I20260709 13:04:35.440812  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:35.376667  9263 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:04:35.442505  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2104) took 1992 ms. Trace:
I20260709 13:04:35.442854  9263 rpcz_store.cc:276] 0709 13:04:33.449920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.369619 (+1919699us) service_pool.cc:224] Handling call
0709 13:04:35.369914 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.442243 (+ 72329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.442452 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.358560  9089 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:35.444614  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 86172.4us
W20260709 13:04:35.373231 12087 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:04:35.454100 11769 raft_consensus.cc:3055] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:35.454563 11769 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:35.454653 12028 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:35.455080 12028 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:35.455482 12028 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:35.417088  9264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1703) took 1965 ms (client timeout 1699 ms). Trace:
W20260709 13:04:35.456136  9264 rpcz_store.cc:269] 0709 13:04:33.414336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.373578 (+1959242us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:35.379341 (+  5763us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:04:35.418814  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.461603 11977 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.421687  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.376091 12023 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:35.425132  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.422968 12067 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.523463  9192 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:35.337872 12074 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:35.441395 11725 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:35.541296  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 188911us
W20260709 13:04:35.542310  9826 leader_election.cc:336] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.942s (SENT)
I20260709 13:04:35.543049  9266 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:04:35.414791  9860 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:04:35.445192 12076 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:04:35.445494  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:35.546044  9270 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2095) took 2343 ms (client timeout 2097 ms). Trace:
I20260709 13:04:35.448554  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1911) took 1949 ms. Trace:
I20260709 13:04:35.450042 12080 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.450891 11728 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:35.463680  9859 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:35.463590  9017 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:35.464249  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:35.466151  9269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1688) took 2070 ms (client timeout 1774 ms). Trace:
W20260709 13:04:35.556607  9269 rpcz_store.cc:269] 0709 13:04:33.395712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.395838 (+   126us) service_pool.cc:224] Handling call
0709 13:04:35.465916 (+2070078us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54000eb5d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:35.466055 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3164569984}
I20260709 13:04:35.467300  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:35.379518 11867 consensus_queue.cc:1059] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 13:04:35.473922 12069 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:35.475868 12082 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:35.477998  9194 raft_consensus.cc:2410] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:04:35.481237  9087 connection.cc:256] RPC call timeout handler was delayed by -1.0249483409999698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
I20260709 13:04:35.562033 11805 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:35.562476 11805 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:35.481758 11980 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:35.482653 12039 raft_consensus.cc:3055] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:04:35.484264  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 249772us
I20260709 13:04:35.489439  9264 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:35.490563  9263 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:35.455566 11769 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.435141 12018 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.573998  9123 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:35.496003  9855 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:35.496778  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.412950  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 834) took 2096 ms. Trace:
I20260709 13:04:35.575798  9862 rpcz_store.cc:276] 0709 13:04:33.316503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.319173 (+  2670us) service_pool.cc:224] Handling call
0709 13:04:33.319532 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:33.493760 (+174228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.380296 (+1886536us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400511654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:35.412893 (+ 32597us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285971840}
I20260709 13:04:35.577144  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.502069 12028 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:35.577909  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:35.578616  9263 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:35.579445 12028 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:35.420899  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2105) took 1970 ms. Trace:
I20260709 13:04:35.580416  9271 rpcz_store.cc:276] 0709 13:04:33.450215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.371505 (+1921290us) service_pool.cc:224] Handling call
0709 13:04:35.413477 (+ 41972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.419682 (+  6205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.419726 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:35.419807 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:35.419863 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.420097 (+   234us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:35.420823 (+   726us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:35.504390 12094 log.cc:927] Time spent T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:04:35.504596  9861 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:04:35.510784  9272 raft_consensus.cc:2410] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:35.585215  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 613) took 2168 ms. Trace:
I20260709 13:04:35.585556  9272 rpcz_store.cc:276] 0709 13:04:33.416933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.280182 (+1863249us) service_pool.cc:224] Handling call
0709 13:04:35.585132 (+304950us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:35.513371  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 50507.4us
I20260709 13:04:35.521421  9857 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:35.588279  9268 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:35.590025  9122 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:35.563273 11805 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.579391  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:35.590937  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.591562  9263 raft_consensus.cc:2410] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:35.591657  9120 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:35.592768  9115 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 13:04:35.593109  9019 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36634 (request call id 339) took 21795 ms (client timeout 15000 ms). Trace:
I20260709 13:04:35.594089 11769 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.595784 12039 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:35.593437  9019 rpcz_store.cc:269] 0709 13:04:13.797626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:13.797917 (+   291us) service_pool.cc:224] Handling call
0709 13:04:35.592876 (+21794959us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:29.403656 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:29.432228 (+ 28572us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:29.432260 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:04:29.432277 (+    17us) tablet.cc:662] Decoding operations
0709 13:04:29.433031 (+   754us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:29.433083 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:29.433116 (+    33us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:04:29.433611 (+   495us) tablet.cc:701] Row locks acquired
0709 13:04:29.433628 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:04:29.433759 (+   131us) write_op.cc:270] Start()
0709 13:04:29.433941 (+   182us) write_op.cc:276] Timestamp: P: 1783602269433719 usec, L: 0
0709 13:04:29.433964 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:04:29.436055 (+  2091us) log.cc:844] Serialized 2872 byte log entry
0709 13:04:30.312135 (+876080us) op_driver.cc:464] REPLICATION: finished
0709 13:04:30.663661 (+351526us) write_op.cc:301] APPLY: starting
0709 13:04:30.668200 (+  4539us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:30.707964 (+ 39764us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:30.725428 (+ 17464us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:30.819541 (+ 94113us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:30.961448 (+141907us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:04:31.057268 (+ 95820us) write_op.cc:312] APPLY: finished
0709 13:04:31.137766 (+ 80498us) log.cc:844] Serialized 59 byte log entry
0709 13:04:31.146114 (+  8348us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:31.153605 (+  7491us) write_op.cc:454] Released schema lock
0709 13:04:31.155059 (+  1454us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:31.192449 (+ 37390us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:31.196426 (+  3977us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":215297,"reported_tablets":26,"spinlock_wait_cycles":20096,"child_traces":[["op",{"apply.queue_time_us":303684,"apply.run_cpu_time_us":5041,"apply.run_wall_time_us":527893,"num_ops":7,"prepare.queue_time_us":771680,"prepare.run_cpu_time_us":7705,"prepare.run_wall_time_us":84335,"raft.queue_time_us":250245,"raft.run_cpu_time_us":22896,"raft.run_wall_time_us":324532,"replication_time_us":839936,"thread_start_us":1076359,"threads_started":4,"wal-append.queue_time_us":110985,"wal-append.run_cpu_time_us":11860,"wal-append.run_wall_time_us":2602732}]]}
I20260709 13:04:35.551681  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:35.546660  9270 rpcz_store.cc:269] 0709 13:04:33.188385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.191809 (+  3424us) service_pool.cc:224] Handling call
0709 13:04:35.530898 (+2339089us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b5400302ad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:35.531438 (+   540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101368704}
I20260709 13:04:35.552985  9826 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.557821  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1913) took 1451 ms. Trace:
W20260709 13:04:35.558553  9238 connection.cc:256] RPC call timeout handler was delayed by -1.157094968000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 13:04:35.558933 11517 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.605964  9190 rpcz_store.cc:276] 0709 13:04:34.106090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.467047 (+1360957us) service_pool.cc:224] Handling call
0709 13:04:35.557769 (+ 90722us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:35.606814 11974 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:35.566443  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:35.607227  9238 connection.cc:256] RPC call timeout handler was delayed by -1.157094968000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 13:04:35.530789 12067 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.567699  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.607614  9269 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.608098  9116 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:35.547788 12074 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:35.610842 11805 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:35.565034 12040 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.611624 12040 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:35.612234 12082 raft_consensus.cc:3055] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:35.612627 12082 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:35.613437 12082 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:35.614321 11728 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.614804 12082 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:35.615927  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 45243us
I20260709 13:04:35.616804  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.619128 11817 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:35.546083 12076 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.573208  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.573510  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.638404  9114 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:35.639057  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1706) took 2224 ms. Trace:
I20260709 13:04:35.639463  9268 rpcz_store.cc:276] 0709 13:04:33.415133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.331661 (+1916528us) service_pool.cc:224] Handling call
0709 13:04:35.588095 (+256434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:35.588687 (+   592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.638906 (+ 50219us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400062a50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:35.639009 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567625984}
I20260709 13:04:35.596696 12039 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.598011  9825 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:35.650801  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 108539us
W20260709 13:04:35.650902  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 53003.2us
I20260709 13:04:35.652081  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.654316  9825 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.598035  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:35.601260  9266 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:35.604112  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.606587 11517 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:35.607559  9238 connection.cc:256] RPC call timeout handler was delayed by -1.137294967999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:04:35.661562  9238 connection.cc:256] RPC call timeout handler was delayed by -1.1336949679999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 13:04:35.662036  9238 connection.cc:256] RPC call timeout handler was delayed by -1.113894968000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
W20260709 13:04:35.662416  9238 connection.cc:256] RPC call timeout handler was delayed by -1.0697949679999965s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
I20260709 13:04:35.567974  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:35.664078  9118 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:35.627061  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.547014  9189 rpcz_store.cc:276] 0709 13:04:33.499647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:33.499879 (+   232us) service_pool.cc:224] Handling call
0709 13:04:35.448354 (+1948475us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540010b654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:35.448492 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2753878400}
W20260709 13:04:35.630546  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.566944  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:35.642527 12040 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.639243  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.663246  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.620127  9017 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 3, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:35.643953  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.645336  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.646546  9269 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:35.668432  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.646214  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.668757  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:35.669242  9853 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:35.670065  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.670639  9857 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:04:35.671110  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 104227us
I20260709 13:04:35.559177 12023 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:35.659471  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.658463  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:35.651487  9826 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.874s (SENT)
I20260709 13:04:35.673038  9861 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:35.667188  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.674613  9861 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:04:35.670809  9862 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:35.672382  9858 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:35.668141  9855 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:04:35.676172  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:35.672816  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:35.676754  9859 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.680728  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1705) took 2265 ms. Trace:
I20260709 13:04:35.681286  9265 rpcz_store.cc:276] 0709 13:04:33.414857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.321379 (+1906522us) service_pool.cc:224] Handling call
0709 13:04:35.330768 (+  9389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:35.331122 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:35.331155 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:35.341686 (+ 10531us) log.cc:844] Serialized 26 byte log entry
0709 13:04:35.546994 (+205308us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:35.608976 (+ 61982us) log.cc:844] Serialized 14 byte log entry
0709 13:04:35.609186 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:35.609253 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:35.637365 (+ 28112us) raft_consensus.cc:1685] finished
0709 13:04:35.637384 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:35.680525 (+ 43141us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b54000fb9d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:35.680659 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25723,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":1203,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":6290,"spinlock_wait_cycles":20848384,"thread_start_us":204917,"threads_started":1,"wal-append.queue_time_us":279694}
I20260709 13:04:35.673096  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:35.685134  9856 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:35.685683  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.685662  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.665403  9825 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.688586  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.690150  9857 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:35.691005  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.691324  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.580416 12068 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:35.647848 12076 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.566088 12086 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.696318  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.697254  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.705778  9119 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:04:35.697750  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:35.702046  9087 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.658s (SENT)
I20260709 13:04:35.646497 12035 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:35.642522 12026 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:35.706874  9087 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.668402 12069 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:35.709226  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:35.712658  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:04:35.708235  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56765.1us
I20260709 13:04:35.691133 12074 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:35.718488  9235 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:35.687695 11974 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:35.714349 12040 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:04:35.714726 11980 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:04:35.714993 11975 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:35.773741 11975 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.774212 11975 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:35.710393 11915 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:04:35.776736 11915 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:35.777225 11915 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:35.778278 11915 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.704035 12039 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:35.685606 11754 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:35.685743 12078 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:35.747303  9118 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
W20260709 13:04:35.751506  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.790475  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:35.691920 12070 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:35.689503 11725 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:35.799537  9118 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:04:35.718312 12062 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:35.739717  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.803872  9122 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:35.803586  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:35.769443  9114 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:35.770342 11974 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.716792  9017 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 4 to 5, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:04:35.687520 11806 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:35.698414 12080 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:35.708019 11977 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:35.672474 11817 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:04:35.703454 12028 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:35.703907 11728 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:35.790750  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 118596us
I20260709 13:04:35.800225 12078 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:04:35.792541  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 194551us
W20260709 13:04:35.837466  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 98026.2us
I20260709 13:04:35.709067 12035 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.719141 11944 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:35.766670  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:35.840502  9826 leader_election.cc:336] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.230s (SENT)
I20260709 13:04:35.844147  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:35.762017  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49466.5us
I20260709 13:04:35.710232 11801 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
W20260709 13:04:35.819819  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 117812us
I20260709 13:04:35.782205 11867 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:35.852685  9857 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:04:35.691792 12023 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.784379 11980 raft_consensus.cc:2764] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:04:35.785683 11970 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:35.819434 11977 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.824728  9017 catalog_manager.cc:5697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 3 to 6, term changed from 5 to 6, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to <none>, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 6 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:35.840458 12075 raft_consensus.cc:3055] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:35.760265 11872 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:04:35.760020 12064 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:04:35.801043 12074 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:35.797610 12070 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:35.843031  9118 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 13:04:35.845201  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:35.883553  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:35.884866  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.817965 11974 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:35.838873  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:35.847645  9087 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.683s (SENT)
I20260709 13:04:35.889635  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.786523 11915 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:35.825830 12032 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:35.791894  9119 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:35.863768 11977 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.822111 11969 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:35.846480 12035 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:35.835578 11754 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:35.844239 12078 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:35.876618  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:35.839812 12080 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:35.853608 12068 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:35.846740 11944 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:35.885779  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 43527.4us
I20260709 13:04:35.892557 12028 raft_consensus.cc:2764] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:04:35.904812 12076 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:35.903486 11926 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.001s
I20260709 13:04:35.908471  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:35.864867  9017 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 4 to 6, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 6 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:04:35.913612  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 65984.5us
W20260709 13:04:35.941238  9087 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.928s (SENT)
I20260709 13:04:35.920696 12071 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:35.869315 12075 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:35.886413  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45923.8us
I20260709 13:04:35.891803 11970 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:35.886897 12070 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:35.892450 11725 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:35.939085  9123 raft_consensus.cc:2368] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6.
W20260709 13:04:35.924249  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 83820.9us
W20260709 13:04:35.962796  9826 leader_election.cc:336] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 3.184s (SENT)
I20260709 13:04:35.963339  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.948314 12112 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:35.950729  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.951017  9017 catalog_manager.cc:5697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:35.936969 11944 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:35.935313 12068 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:35.964108  9116 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:35.943872 12080 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.967536  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:35.968067 12023 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:35.968828 12068 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:35.964973 12112 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:35.970429 12112 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.971653 12112 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:35.972551 12099 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:04:35.935663 12026 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:04:35.977612 12069 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:35.978360 12069 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:35.980542 12075 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:35.982213 12075 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:35.983911 12053 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:35.988807 11825 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:35.992157 12069 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:35.992656 11867 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:35.989392 12112 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:04:36.000262 12086 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:35.927484 11980 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:04:36.004104  9859 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
W20260709 13:04:36.004830  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:36.008381  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59769.4us
I20260709 13:04:36.009438  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.006400  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:36.007794 12074 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.028833 12067 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.031114  9857 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:36.032162  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.032117 11754 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.034229 11975 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.033169 12067 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:35.959671 11970 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.028872 12076 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:36.036262 11970 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:36.037247 11975 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:36.042156  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:35.968773  9017 catalog_manager.cc:5697] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:35.960427 11517 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:36.047322  9017 catalog_manager.cc:5697] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 3, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 3 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:04:36.050488  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 44214.9us
I20260709 13:04:36.050725 12084 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:36.051391 12084 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:36.051927 12026 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:36.052438 12026 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:36.063024  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 112373us
I20260709 13:04:36.064272  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:35.945468 11801 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 13:04:36.095323  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:36.102730  9123 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:36.100967  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:36.099973  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 210886us
I20260709 13:04:36.052582 12084 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:36.113153  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 53589.7us
W20260709 13:04:36.124761  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 150862us
I20260709 13:04:36.114529  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:36.112295  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48150.4us
I20260709 13:04:36.125417  9164 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:36.137584  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:36.143555 11517 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:36.144300 11517 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:36.145536 11517 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.146621  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:36.136564  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.149199  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:36.116140  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.150918  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.160681  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.161615  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:36.165059  9263 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:36.168222  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.168833  9263 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:36.169102  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.170634 11975 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:36.171231 11975 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:36.171733 11975 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:36.171896  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.172849  9263 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:36.177155 12018 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:36.178189 12018 consensus_queue.cc:1243] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:36.180469  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.181509  9263 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:36.183425  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.183656  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.188191  9198 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:36.188961  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.189432  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:36.189786  9267 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:04:36.190861  9826 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.548s (SENT)
I20260709 13:04:36.194751  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.195540  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:36.197304  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.200798  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:36.201011  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:36.201838  9264 raft_consensus.cc:2376] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5. Re-sending same reply.
I20260709 13:04:36.202260  9263 raft_consensus.cc:2376] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5. Re-sending same reply.
I20260709 13:04:36.200794  9123 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:36.206734  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:36.207301  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.207453  9268 raft_consensus.cc:2376] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4. Re-sending same reply.
I20260709 13:04:36.207984  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.210932  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:36.208276  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.209344  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:36.094964 12114 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:36.070130 11915 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:04:36.210958  9017 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 2 to 3, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 3 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.220480  9193 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:36.220494  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.219309  9828 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.224805  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.053318 12026 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.118408 12071 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:36.230626  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.111685 12113 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:36.231339  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.233271  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.234151  9268 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:36.235313  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.238587  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.239627  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:36.240115  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 521673us
I20260709 13:04:36.240625  9263 raft_consensus.cc:2410] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:04:36.239617  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.242377  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:36.244035  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:36.226969 11769 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.002s
I20260709 13:04:36.249490  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.249922  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.246007  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:36.246856  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.113341 12023 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 13:04:36.250103  9268 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:36.229849 11980 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:36.231904 12067 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:36.140467 11825 consensus_queue.cc:1059] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 13:04:36.113413 11725 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
I20260709 13:04:36.260995  9197 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:36.261605  9195 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:04:36.248551  9235 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:36.257925  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.263485  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:36.264101  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.264384  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.278661  9270 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:36.279011  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.267302  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.207860 12064 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:04:36.227361 11806 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:36.259212  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:36.270510  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 144803us
I20260709 13:04:36.237219 12113 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.211779 12028 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:04:36.272367  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:36.277740  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.262265 12069 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:36.279619  9861 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:36.288270  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.222522 12104 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:04:36.277788  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.293546  9853 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:36.296396  9197 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:04:36.280535  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:36.308854  9197 pending_rounds.cc:85] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 6
I20260709 13:04:36.260107 12115 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:36.267225 11980 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:36.246122 12035 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.309355  9197 pending_rounds.cc:107] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:04:36.282912 11801 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:36.301499  9017 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 3 to 5, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:36.312340  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.301266  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.314888  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.317790  9197 raft_consensus.cc:2889] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:36.310011  9192 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:36.315840 12069 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:36.324467 12028 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:36.324565  9193 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:36.325603  9196 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:04:36.291688 11769 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:36.290007 11915 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.299876 12071 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:36.300678 12113 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:36.306795 11969 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:36.346192  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.318343 11980 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:36.318107 12067 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:36.353166  9122 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:36.314098 12115 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:36.364996 12115 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:36.366343 12035 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:36.356647  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:36.368708  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.351230  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:36.329435 11801 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:36.361034  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:36.360726 12053 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:36.364362  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:36.369771  9858 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:36.373682  9116 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:36.356048  9017 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 4 to 6, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 6 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:36.372144  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:36.384835  9191 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:36.374043  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:36.359297 12082 consensus_meta.cc:287] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.744s	user 0.001s	sys 0.017s
I20260709 13:04:36.385872  9197 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:36.316193 11975 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.361827 12071 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:36.358853 12074 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:36.359074 11915 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:36.476842  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 140174us
I20260709 13:04:36.361280 12068 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.424687 11806 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:36.483366  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 207807us
W20260709 13:04:36.447685  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.484246  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:36.484735  9855 pending_rounds.cc:85] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 4
I20260709 13:04:36.485075  9855 pending_rounds.cc:107] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:04:36.485318  9855 raft_consensus.cc:2889] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:36.485363  9235 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:36.406682  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.487258  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:36.382128 11801 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:36.491291  9235 leader_election.cc:400] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6.
I20260709 13:04:36.491976  9235 leader_election.cc:403] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:04:36.492326  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.491915  9017 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 1 to 4, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 4 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:36.431327 12069 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.480054  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:36.481549  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.482964  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:36.489290  9193 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:36.520123  9265 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:36.520823  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:36.493131  9195 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:36.463459 11769 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:36.491406  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:36.525313  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.496129  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:36.528115  9192 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:36.497965  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:36.533509  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:36.538210  9270 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:36.496670  9271 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
W20260709 13:04:36.535058  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.470026  9194 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:36.490621  9197 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:36.490927 12028 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:36.495620 12023 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:36.482514 12018 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.522135 12104 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.547727  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:36.609113  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 61479.4us
I20260709 13:04:36.524070 12064 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:36.549209  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 267030us
W20260709 13:04:36.545603 12087 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:04:36.495854 11754 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.545926 11725 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.551199  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:36.546242 12028 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:36.615844 12098 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:04:36.630187 12075 consensus_meta.cc:287] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.647s	user 0.013s	sys 0.001s
I20260709 13:04:36.631873  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:04:36.685047  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 53294.4us
I20260709 13:04:36.610006  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:36.687951  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 78054.8us
I20260709 13:04:36.688612  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:36.641374  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:36.688992  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:36.629249 12126 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:04:36.642467  9197 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
W20260709 13:04:36.664129  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 113224us
I20260709 13:04:36.550462 12067 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:36.630065 12124 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
W20260709 13:04:36.689251  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 154420us
W20260709 13:04:36.606801 11974 consensus_meta.cc:287] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.000s
I20260709 13:04:36.619328 12023 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:36.691195  9089 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:36.691828  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 208952us
I20260709 13:04:36.674407 12028 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:36.692267  9198 raft_consensus.cc:2410] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 13:04:36.701606  9857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.000s	sys 0.003s
W20260709 13:04:36.763051  9857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.000s	sys 0.011s
I20260709 13:04:36.749819  9192 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
W20260709 13:04:36.740731  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 51813.9us
I20260709 13:04:36.764262  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:36.616954 12018 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.627220 12104 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.690205 11805 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:36.739894 11969 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:36.668296 11754 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:36.673056 11725 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:36.795172  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 426516us
I20260709 13:04:36.811617  9196 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:36.797854  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:36.814123  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:36.774820 12080 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:36.815623  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:36.790728 11969 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:36.796811  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 108164us
I20260709 13:04:36.757789 11974 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.833160  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:36.854295 12119 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.308s	user 0.000s	sys 0.007s
I20260709 13:04:36.812755 11769 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:36.860698  9860 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:36.795351 11817 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
W20260709 13:04:36.836213 12039 consensus_meta.cc:287] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.046s	user 0.011s	sys 0.001s
W20260709 13:04:36.845446 11944 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.002s
W20260709 13:04:36.855273 12070 consensus_meta.cc:287] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.886s	user 0.010s	sys 0.001s
W20260709 13:04:36.856665 12078 consensus_meta.cc:287] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.908s	user 0.011s	sys 0.000s
I20260709 13:04:36.838287 11969 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:36.870023  9087 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:36.847389 12116 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:36.860155 11977 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.928s	user 0.011s	sys 0.000s
I20260709 13:04:36.871940  9117 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:36.855922 11915 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:36.867918 11975 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:36.870636 12071 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:36.870455 12068 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:36.930497 12153 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:36.911893 11977 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.878365 12131 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:36.881558 12127 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.419s	user 0.000s	sys 0.001s
I20260709 13:04:36.856858 12115 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.879267 12039 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:04:36.948410  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 199883us
I20260709 13:04:36.864696 11769 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.963864  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:36.874110 11944 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:36.970526  9859 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:36.878898 12070 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:36.985028  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 293893us
I20260709 13:04:36.916265 12142 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:36.950677 12131 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:36.953572 11915 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:36.959609 12074 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:04:36.990003  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 206017us
I20260709 13:04:36.877601 12078 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.000730  9267 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:36.962540 12039 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:37.002112  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:36.970933 11975 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.020955  9120 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:36.967970 11974 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:37.023774  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:36.877306 12023 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:36.993492 12131 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:37.043944  9121 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
W20260709 13:04:37.042666  9190 consensus_meta.cc:287] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.004s
I20260709 13:04:37.045696  9190 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:36.993981 11915 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:37.056643  9191 consensus_meta.cc:287] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.001s
I20260709 13:04:37.000156 12147 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.063370  9191 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:37.024339 12039 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:04:37.069202 12133 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
I20260709 13:04:37.070169 11977 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:37.026799 12018 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:37.078241 11970 consensus_meta.cc:287] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.041s	user 0.006s	sys 0.004s
I20260709 13:04:37.081004 12035 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:04:37.082346 11980 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.709s	user 0.005s	sys 0.002s
W20260709 13:04:37.088461 12053 consensus_meta.cc:287] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.649s	user 0.008s	sys 0.000s
I20260709 13:04:37.092737 12139 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:37.093964  9828 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:37.100064 12146 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:04:37.100446 12152 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:37.100829 12120 log.cc:927] Time spent T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
I20260709 13:04:37.023758 12142 raft_consensus.cc:697] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:37.019495 12114 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:37.118664  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:37.117487  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:37.143522  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.183444 12104 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:37.031565 12134 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:37.186367  9119 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:04:37.066819 11915 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:37.207605  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:37.210966  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 117087us
I20260709 13:04:37.246102  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:37.114595 11974 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.115319  9115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.004s	sys 0.020s
W20260709 13:04:37.247074  9115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.006s	sys 0.021s
I20260709 13:04:37.199892 11725 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:37.220423 12141 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.384s	user 0.001s	sys 0.000s
I20260709 13:04:37.206979 12053 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:37.206650 11754 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:37.242341  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 124987us
I20260709 13:04:37.304670 12139 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.216351 11977 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:37.179415 11970 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:37.308235  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 62234.8us
I20260709 13:04:37.244100  9856 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:37.102926 12147 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:37.303083 12130 log.cc:927] Time spent T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:04:37.114238 11874 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
W20260709 13:04:37.286985 12028 consensus_meta.cc:287] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.003s
I20260709 13:04:37.207141 11980 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:37.361030 12151 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
W20260709 13:04:37.363018 12162 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:04:37.370949  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 855) took 1019 ms. Trace:
I20260709 13:04:37.485908  9862 rpcz_store.cc:276] 0709 13:04:36.351377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.420764 (+ 69387us) service_pool.cc:224] Handling call
0709 13:04:36.453398 (+ 32634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.476301 (+ 22903us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:36.672720 (+196419us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.014173 (+341453us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.030036 (+ 15863us) log.cc:844] Serialized 26 byte log entry
0709 13:04:37.030382 (+   346us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:37.030453 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.030627 (+   174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.368940 (+338313us) spinlock_profiling.cc:243] Waited 7.44 ms on lock 0x7b1800272ec0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:37.370671 (+  1731us) raft_consensus.cc:1685] finished
0709 13:04:37.370757 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.370889 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34970,"prepare.queue_time_us":31913,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":345655,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":30242,"spinlock_wait_cycles":17104128,"thread_start_us":370909,"threads_started":2,"wal-append.queue_time_us":287886}
I20260709 13:04:37.488644  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:37.489353  9862 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:37.375877 12115 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:37.378368  9264 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:37.242249 12074 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:37.382143  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.325173 12067 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:37.326067 11974 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:37.386065  9117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2073) took 1816 ms (client timeout 2287 ms). Trace:
I20260709 13:04:37.394219 12152 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:37.394973 12039 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:37.398825 11915 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:37.406262  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2000) took 1080 ms. Trace:
W20260709 13:04:37.382153  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:37.307520  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:37.315901 12150 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:37.421754  9861 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
W20260709 13:04:37.356823 12129 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 13:04:37.344206 12125 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:37.453111  9828 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:37.349597 12155 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:37.476742  9270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.005s	sys 0.014s
W20260709 13:04:37.596699  9270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.006s	sys 0.015s
I20260709 13:04:37.595513  9862 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:37.363111 12108 raft_consensus.cc:3055] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:04:37.599009  9826 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.016s	sys 0.012s
I20260709 13:04:37.330396 12149 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:37.456334 12146 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:04:37.452538 11977 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.561000 12147 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:37.490681 12028 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.564023 12067 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:04:37.567878 11801 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.044s	user 0.012s	sys 0.000s
W20260709 13:04:37.572005  9117 rpcz_store.cc:269] 0709 13:04:35.570045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.570208 (+   163us) service_pool.cc:224] Handling call
0709 13:04:37.385868 (+1815660us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400266e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:37.385997 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2836834688}
I20260709 13:04:37.548719  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 644) took 1305 ms. Trace:
W20260709 13:04:37.485523  9238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.000s	sys 0.020s
I20260709 13:04:37.451984 11725 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:37.587216  9118 rpcz_store.cc:276] 0709 13:04:36.326353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.327393 (+  1040us) service_pool.cc:224] Handling call
0709 13:04:36.327839 (+   446us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:36.816137 (+488298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.816172 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:36.817573 (+  1401us) log.cc:844] Serialized 52 byte log entry
0709 13:04:37.006734 (+189161us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:37.243908 (+237174us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.244204 (+   296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.244303 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.406027 (+161724us) raft_consensus.cc:1685] finished
0709 13:04:37.406046 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.406199 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24227,"prepare.queue_time_us":360742,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":115384,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":54024,"thread_start_us":795822,"threads_started":3,"wal-append.queue_time_us":558652}
W20260709 13:04:37.589039  9161 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.014s	sys 0.027s
I20260709 13:04:37.588802  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:37.485216 12070 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:37.592172  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 890) took 1540 ms. Trace:
W20260709 13:04:37.587193 12064 consensus_meta.cc:287] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.954s	user 0.006s	sys 0.005s
W20260709 13:04:37.606398  9238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.000s	sys 0.022s
W20260709 13:04:37.620316  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 856144us
W20260709 13:04:37.598212  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 145185us
I20260709 13:04:37.607884  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:37.621202  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:37.610489  9266 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
W20260709 13:04:37.599704  9826 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.016s	sys 0.012s
I20260709 13:04:37.572414 12167 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.626690  9198 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:37.611577  9857 rpcz_store.cc:276] 0709 13:04:36.051723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.051924 (+   201us) service_pool.cc:224] Handling call
0709 13:04:36.052290 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.052629 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.052659 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.053138 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.772946 (+719808us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:36.772998 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.773085 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.591967 (+818882us) raft_consensus.cc:1685] finished
0709 13:04:37.591989 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.592116 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2163,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":38031,"thread_start_us":709975,"threads_started":1,"wal-append.queue_time_us":959830}
I20260709 13:04:37.593900 12125 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.598119 12150 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:37.456849 12139 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:37.593550 12113 consensus_meta.cc:287] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.221s	user 0.011s	sys 0.000s
I20260709 13:04:37.606316  9268 rpcz_store.cc:276] 0709 13:04:36.243686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.274405 (+ 30719us) service_pool.cc:224] Handling call
0709 13:04:36.291400 (+ 16995us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.301962 (+ 10562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.301994 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.312688 (+ 10694us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.547627 (+234939us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:36.547666 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.547731 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.548457 (+1000726us) raft_consensus.cc:1685] finished
0709 13:04:37.548479 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.548657 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10608,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":24370,"thread_start_us":192763,"threads_started":1,"wal-append.queue_time_us":439887}
W20260709 13:04:37.622627  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 625759us
I20260709 13:04:37.607525  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:37.603704 11977 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:37.607816  9161 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.014s	sys 0.027s
W20260709 13:04:37.611225 11969 consensus_meta.cc:287] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.000s
W20260709 13:04:37.602839  9265 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.012s	sys 0.033s
W20260709 13:04:37.677157  9265 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.012s	sys 0.033s
I20260709 13:04:37.629880  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:37.604802  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 1997) took 1279 ms. Trace:
I20260709 13:04:37.631374  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:37.596560 12155 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:37.599335 12108 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:37.606201 11915 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:37.636080 12135 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.495s	user 0.002s	sys 0.004s
I20260709 13:04:37.630613  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:37.690430  9264 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:04:37.636678 12148 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.377s	user 0.001s	sys 0.000s
I20260709 13:04:37.636366 12067 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.497277 12142 consensus_queue.cc:237] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:37.639003 11974 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:37.672501  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 905533us
I20260709 13:04:37.618762 12082 raft_consensus.cc:2764] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:04:37.605929 12032 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:37.712415 12082 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.713112 12082 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.715363 12082 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:37.686528  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 56715.8us
I20260709 13:04:37.716341  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:37.716539 12040 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:37.678367  9115 rpcz_store.cc:276] 0709 13:04:36.325324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.327159 (+  1835us) service_pool.cc:224] Handling call
0709 13:04:36.353833 (+ 26674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.354037 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:36.373264 (+ 19227us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.604174 (+1230910us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.604446 (+   272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:37.604489 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:37.604547 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.604622 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.604749 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66796,"prepare.queue_time_us":92078,"prepare.run_cpu_time_us":142,"prepare.run_wall_time_us":4816,"raft.queue_time_us":150985,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":21148,"thread_start_us":1106799,"threads_started":2}
I20260709 13:04:37.719923 12172 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:37.687721  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 75841.3us
I20260709 13:04:37.720230 11970 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:37.720508 12172 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.688019  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 853) took 1494 ms. Trace:
I20260709 13:04:37.723382 12172 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:37.640297 11980 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:37.625402 11801 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:37.638648 12028 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:37.747449  9862 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:04:37.631858 12125 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:37.695878  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 862) took 1212 ms. Trace:
I20260709 13:04:37.619907 11725 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:37.687747 12023 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:37.761579 11974 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:37.761274 12108 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.762202 11974 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.688787 12155 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:37.764024 12039 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:37.688978 11969 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:37.766767 11969 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:37.723999 12160 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.768069 12160 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.769165  9269 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:37.770375 12177 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.770560 11769 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:37.770726 12160 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:37.720983 12136 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:37.770908 12177 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.771833 12136 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.773237 12177 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:37.628597 12064 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6.
I20260709 13:04:37.698145 12069 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:37.774757 12076 raft_consensus.cc:3055] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:37.775265 12076 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:37.774224 12136 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:37.726007 12122 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:37.734959  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:37.728300  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2131) took 1245 ms. Trace:
I20260709 13:04:37.778331  9265 rpcz_store.cc:276] 0709 13:04:36.483262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.545291 (+ 62029us) service_pool.cc:224] Handling call
0709 13:04:36.547463 (+  2172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.570680 (+ 23217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.626407 (+ 55727us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.675161 (+ 48754us) log.cc:844] Serialized 26 byte log entry
0709 13:04:37.677723 (+1002562us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:37.677782 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.677878 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.728044 (+ 50166us) raft_consensus.cc:1685] finished
0709 13:04:37.728068 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.728230 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120907,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":23321,"thread_start_us":892323,"threads_started":1,"wal-append.queue_time_us":935722}
I20260709 13:04:37.724041 12139 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:37.701076 12131 consensus_meta.cc:287] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.633s	user 0.006s	sys 0.003s
I20260709 13:04:37.746923  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:37.695372 12062 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:37.753834  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 128830us
I20260709 13:04:37.755259  9855 rpcz_store.cc:276] 0709 13:04:36.483704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.484197 (+   493us) service_pool.cc:224] Handling call
0709 13:04:36.484507 (+   310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:36.486039 (+  1532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.486072 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:36.486695 (+   623us) log.cc:844] Serialized 52 byte log entry
0709 13:04:36.688831 (+202136us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:36.744595 (+ 55764us) log.cc:844] Serialized 14 byte log entry
0709 13:04:36.765804 (+ 21209us) log.cc:844] Serialized 14 byte log entry
0709 13:04:37.362630 (+596826us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b48001bc258. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:04:37.392796 (+ 30166us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b18003c9ca0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535c52a6 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1
0709 13:04:37.393247 (+   451us) spinlock_profiling.cc:243] Waited 5.22 ms on lock 0x7b18003c9ca0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:37.400753 (+  7506us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b18003c9ca0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f765628c4bc 00007f76535da48b 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287
0709 13:04:37.400891 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.400968 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.695639 (+294671us) raft_consensus.cc:1685] finished
0709 13:04:37.695657 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.695816 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61237,"prepare.queue_time_us":683600,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":118,"raft.queue_time_us":602482,"raft.run_cpu_time_us":12885,"raft.run_wall_time_us":652074,"spinlock_wait_cycles":107586816,"thread_start_us":303447,"threads_started":2,"wal-append.queue_time_us":344268}
I20260709 13:04:37.759397 12156 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:37.759665 12183 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:04:37.760455 11725 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:37.760068 12150 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.762674 12108 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:37.774917 12069 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.775904  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 854) took 1424 ms. Trace:
I20260709 13:04:37.688750  9268 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:37.723316  9854 rpcz_store.cc:276] 0709 13:04:36.193332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.193483 (+   151us) service_pool.cc:224] Handling call
0709 13:04:36.193892 (+   409us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:36.194358 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.194397 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:36.195038 (+   641us) log.cc:844] Serialized 52 byte log entry
0709 13:04:36.327047 (+132009us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:36.364603 (+ 37556us) log.cc:844] Serialized 14 byte log entry
0709 13:04:36.954501 (+589898us) spinlock_profiling.cc:243] Waited 6.37 ms on lock 0x7b180001d080. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287 00007f76566ff208
0709 13:04:37.199323 (+244822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.199400 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.669830 (+470430us) raft_consensus.cc:1685] finished
0709 13:04:37.687828 (+ 17998us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.687966 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16663,"prepare.queue_time_us":33575,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":69631,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":207,"spinlock_wait_cycles":14651264,"thread_start_us":376295,"threads_started":2,"wal-append.queue_time_us":297458}
I20260709 13:04:37.778359  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:37.783176  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:37.605937 12166 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:37.791352  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:37.825399  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:37.836261  9087 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.578s (SENT)
I20260709 13:04:37.940392  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:37.916772  9264 raft_consensus.cc:2393] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:37.942447  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:37.926358  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:37.748265 12173 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.768748 11801 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:37.931685  9272 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:37.632251 12167 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.921535  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:37.944239 11969 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.686201 12113 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.776259 12076 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:37.936256  9858 rpcz_store.cc:276] 0709 13:04:36.351145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.448612 (+ 97467us) service_pool.cc:224] Handling call
0709 13:04:36.449036 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:36.478864 (+ 29828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:36.478897 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:36.520676 (+ 41779us) log.cc:844] Serialized 26 byte log entry
0709 13:04:36.852528 (+331852us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:36.852584 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:36.852684 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:37.775658 (+922974us) raft_consensus.cc:1685] finished
0709 13:04:37.775676 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:37.775844 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":302159,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":25530,"thread_start_us":238772,"threads_started":1,"wal-append.queue_time_us":638256}
W20260709 13:04:38.001142  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.036212  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:38.030553  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 194343us
I20260709 13:04:38.032260  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:37.792150 12136 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:37.972391  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.049891  9264 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:04:38.033064  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 254768us
I20260709 13:04:37.957412  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:37.953944  9265 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:37.953112  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:37.917522 12064 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.035959  9198 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:38.043509  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 850) took 1877 ms. Trace:
I20260709 13:04:38.045298  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:38.045418  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 347046us
I20260709 13:04:38.035734  9196 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:04:38.046295  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.060772  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:38.061084  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:38.062193  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.061829  9263 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:37.938974 12166 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.953444  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:38.052830  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 99834.1us
I20260709 13:04:38.058772  9859 rpcz_store.cc:276] 0709 13:04:36.165732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.166728 (+   996us) service_pool.cc:224] Handling call
0709 13:04:36.167135 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:36.983468 (+816333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.043201 (+1059733us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b5400506254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:38.043432 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847229696}
I20260709 13:04:37.991313 11725 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:37.991997 12159 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.064253  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:37.991047 12149 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:37.951365 12139 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:38.064996  9189 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:38.062482  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:38.060006 12152 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:37.936468 11980 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:37.945382 12023 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:37.917173 12131 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:37.983891 12028 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.064021  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:38.114919  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 50982.5us
I20260709 13:04:38.116022  9161 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:38.069315  9856 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:38.045351  9087 leader_election.cc:336] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.699s (SENT)
I20260709 13:04:38.069831 11725 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:37.924854 12156 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:38.063545 12064 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:38.075481  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.123560  9263 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:38.066154  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.069680 12166 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:38.068607 12076 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:38.075613 12159 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.084743  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.086423 12167 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:38.057019 12179 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:38.089747  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.143309  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.143688  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.015537 12185 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:04:37.937215 12062 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:38.121695  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 76376.3us
I20260709 13:04:38.123356 12158 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:38.127936  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.130445  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.133453  9268 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:38.134266  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:38.136821  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.139672  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:38.195667  9122 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:38.150872  9191 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:38.153882  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:38.156149  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 83936.2us
I20260709 13:04:38.076671 12115 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:04:38.157076  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:38.198042  9123 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:04:38.158931  9087 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.710s (SENT)
I20260709 13:04:38.198683  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:38.077726 11974 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.205569  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:38.206815  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.207366  9263 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
W20260709 13:04:38.208886  9861 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 852) took 1858 ms (client timeout 1899 ms). Trace:
I20260709 13:04:38.209534 12115 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:38.209915 12115 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:38.210005  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2078) took 2533 ms. Trace:
I20260709 13:04:38.210669 12115 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:38.211246  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2125) took 1732 ms (client timeout 1804 ms). Trace:
W20260709 13:04:38.211640  9272 rpcz_store.cc:269] 0709 13:04:36.478442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.479941 (+  1499us) service_pool.cc:224] Handling call
0709 13:04:38.211088 (+1731147us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54002a3354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:38.211188 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2850486656}
W20260709 13:04:38.212599 12128 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.451s	user 0.004s	sys 0.001s
I20260709 13:04:38.212765  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.213861  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.214619 12131 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.214815  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.216812 12032 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:38.216923  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 58043.3us
I20260709 13:04:38.121722 12028 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.217682  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.218989  9263 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:38.162638 12156 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.162891  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:38.220742  9271 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:38.163470 12062 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.163990  9854 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:38.221758 12078 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:38.222705  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:38.223666 11975 raft_consensus.cc:3055] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:38.224116 11975 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:38.224782 11944 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:38.224987 11975 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.226166 11975 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:38.226636 12187 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.463s	user 0.000s	sys 0.001s
I20260709 13:04:38.168004 12155 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.227564  9194 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:38.228576  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2134) took 1744 ms. Trace:
I20260709 13:04:38.229385 12155 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.162638 12064 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:38.228960  9270 rpcz_store.cc:276] 0709 13:04:36.483984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.588589 (+104605us) service_pool.cc:224] Handling call
0709 13:04:36.625749 (+ 37160us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:37.597104 (+971355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.597135 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:37.599005 (+  1870us) log.cc:844] Serialized 52 byte log entry
0709 13:04:37.742379 (+143374us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:37.742435 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:37.742544 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.226126 (+483582us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:38.228284 (+  2158us) raft_consensus.cc:1685] finished
0709 13:04:38.228304 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.228520 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":167780,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":20289,"spinlock_wait_cycles":52817280,"thread_start_us":1081995,"threads_started":2,"wal-append.queue_time_us":152009}
I20260709 13:04:38.114741 12023 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.232086  9270 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:04:38.233042  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1920) took 2120 ms. Trace:
I20260709 13:04:38.233215 12023 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:38.233429  9189 rpcz_store.cc:276] 0709 13:04:36.112438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.112549 (+   111us) service_pool.cc:224] Handling call
0709 13:04:36.212542 (+ 99993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.077010 (+1864468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.232806 (+155796us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54000677d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:38.232952 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4241327872}
I20260709 13:04:38.234647  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.235164  9189 raft_consensus.cc:2376] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4. Re-sending same reply.
I20260709 13:04:38.223430 11974 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.236171 11974 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.237308  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:38.237601  9272 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:38.237929  9123 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:04:38.239347  9121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 487) took 2549 ms (client timeout 2098 ms). Trace:
I20260709 13:04:38.194743  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:38.239804  9121 rpcz_store.cc:269] 0709 13:04:35.689849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.690762 (+   913us) service_pool.cc:224] Handling call
0709 13:04:38.239170 (+2548408us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400253d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:38.239271 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3109811456}
W20260709 13:04:38.240193  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 45573.3us
W20260709 13:04:38.241017  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 628) took 2048 ms (client timeout 1632 ms). Trace:
W20260709 13:04:38.241421  9267 rpcz_store.cc:269] 0709 13:04:36.192458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.194490 (+  2032us) service_pool.cc:224] Handling call
0709 13:04:38.240815 (+2046325us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b54000f9bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:38.240938 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1851668864}
I20260709 13:04:38.242851  9267 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:38.113588 11980 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:38.243225 11974 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:38.244939 11980 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:38.245947 11517 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:38.246357 11517 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.246690 11517 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:38.248211  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 863) took 1764 ms. Trace:
I20260709 13:04:38.248575  9860 rpcz_store.cc:276] 0709 13:04:36.484073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.491938 (+  7865us) service_pool.cc:224] Handling call
0709 13:04:38.248043 (+1756105us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b540009ae54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:38.248160 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729796992}
I20260709 13:04:38.249541  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.250594  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.250916  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.205617  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.252020  9195 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:38.171963 12186 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:38.174163 12069 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.124172 12082 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:38.254518  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.177870  9115 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:38.076886 12149 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.257017 12149 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.199927  9193 raft_consensus.cc:2368] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6.
W20260709 13:04:38.211365 11977 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.519s	user 0.006s	sys 0.000s
I20260709 13:04:38.215495  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:38.215572  9190 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1922) took 2102 ms (client timeout 1995 ms). Trace:
W20260709 13:04:38.275312  9190 rpcz_store.cc:269] 0709 13:04:36.113146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.113326 (+   180us) service_pool.cc:224] Handling call
0709 13:04:38.215414 (+2102088us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b5400398d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:38.215509 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506636032}
I20260709 13:04:38.128013 11725 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:38.161635  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:38.279047  9116 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:38.210393  9120 rpcz_store.cc:276] 0709 13:04:35.676363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.676543 (+   180us) service_pool.cc:224] Handling call
0709 13:04:35.708574 (+ 32031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.021608 (+1313034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.209802 (+1188194us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400255154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:38.209952 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3017742208}
I20260709 13:04:38.216493 12159 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:38.209312  9861 rpcz_store.cc:269] 0709 13:04:36.350499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.350623 (+   124us) service_pool.cc:224] Handling call
0709 13:04:38.208721 (+1858098us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54007323d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:38.208827 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2389477760}
I20260709 13:04:38.169046 12113 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:38.255234 12064 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:38.257941 12115 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:38.252041 12028 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:38.257158 12125 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:38.223675  9122 raft_consensus.cc:2410] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:04:38.258898  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.257870  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.260859  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:38.262351  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.262925  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.302920  9191 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:38.264127  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:38.264669  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 45205.9us
I20260709 13:04:38.304399  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:38.305457  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:38.263535  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.237824  9271 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:38.228609  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.303604  9195 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:38.255875 12200 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:38.181474 12179 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:38.267400  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 215710us
I20260709 13:04:38.239634  9856 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:04:38.268357  9854 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:38.270048  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:38.273952  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:38.284795  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:38.275035 12023 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:38.287971 12064 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.266700  9270 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:38.313309  9860 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:38.332669  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:38.340112  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:38.362905  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:38.317741  9828 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:38.365433  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 95501.7us
I20260709 13:04:38.366760  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:38.341439  9857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.006s	sys 0.014s
I20260709 13:04:38.367645  9858 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
W20260709 13:04:38.368194  9857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.006s	sys 0.015s
I20260709 13:04:38.344532  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2130) took 1861 ms. Trace:
I20260709 13:04:38.368831  9266 rpcz_store.cc:276] 0709 13:04:36.482991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.498751 (+ 15760us) service_pool.cc:224] Handling call
0709 13:04:36.538022 (+ 39271us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:37.622178 (+1084156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.344190 (+722012us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540032e454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:38.344477 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2465570688}
I20260709 13:04:38.348174  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2133) took 1864 ms. Trace:
I20260709 13:04:38.348384  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:38.352653  9855 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:04:38.345368 12131 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:38.344331  9859 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:38.370357  9269 rpcz_store.cc:276] 0709 13:04:36.483734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:36.525503 (+ 41769us) service_pool.cc:224] Handling call
0709 13:04:36.525799 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:37.769591 (+1243792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.347946 (+578355us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54001c4dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:38.348117 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2858976128}
W20260709 13:04:38.380851  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72090.5us
I20260709 13:04:38.330152 12108 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:04:38.382496  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49942.1us
I20260709 13:04:38.391084 12071 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:38.395411  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:38.372697 12180 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:04:38.381886 12156 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.395116  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 77460.7us
I20260709 13:04:38.400146  9271 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:04:38.395116  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:38.401288  9120 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:38.348234 12076 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:38.397986  9116 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:38.398569  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:38.404224  9121 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:38.404047  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:38.405117  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:38.283641 11977 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.382189 12152 raft_consensus.cc:2764] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:04:38.361737 12062 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.406399  9198 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:04:38.328953 12053 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:04:38.405138  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:38.371910 12078 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.396838 12075 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:38.411026  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:38.405856  9123 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:38.417171  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:38.328817 11806 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:04:38.348585 12200 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.400542 12150 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:38.429311 12150 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.429719 12150 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:38.430356  9120 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:04:38.373188 12023 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:38.399570 12067 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:38.408286 12108 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:38.423767  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:38.397372 12069 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:38.426501  9122 raft_consensus.cc:2410] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:38.428541  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:38.428611 12200 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:38.419979  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:38.433218  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:38.416921 12076 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:38.451824  9121 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:38.399220 12082 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:04:38.451987  9117 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:38.408258 12028 consensus_queue.cc:1059] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:04:38.420873 12075 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:38.410344 12156 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:38.463712  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:38.409648 12122 consensus_queue.cc:1059] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:04:38.468611 11867 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:38.469950 11977 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.472543 12134 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:38.426728 12074 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:04:38.472743  9269 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:38.472321 12108 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:38.478992  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 117656us
W20260709 13:04:38.496665  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 45199.8us
I20260709 13:04:38.469961 12158 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.570282  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0001234660000478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
I20260709 13:04:38.545403  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.463248 12069 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.471374 11517 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.421960 12062 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.552289 12076 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:38.553797 12064 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:38.555816 11801 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:38.557576 12156 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.561152 12122 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:04:38.561339 12192 log.cc:927] Time spent T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Append to log took a long time: real 0.129s	user 0.001s	sys 0.001s
W20260709 13:04:38.563241 12196 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:04:38.474027 12203 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:04:38.437839 12121 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:04:38.564495  9118 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:38.425153 12078 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.570812  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:38.544776  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 66081.3us
I20260709 13:04:38.564013 12142 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:38.497201 12075 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:38.489962 12172 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.009s	sys 0.006s
I20260709 13:04:38.580945  9196 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:38.584928  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:38.564241 11867 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:38.565507 12082 consensus_queue.cc:1243] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:38.564051 12134 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:38.603024  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:38.603113  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 399543us
I20260709 13:04:38.603420  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:38.604853  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 209487us
I20260709 13:04:38.613827  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.592068 12166 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.581496 12158 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.745893  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.614472  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.696688  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.595956 12032 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.613993  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:38.822902  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 252156us
I20260709 13:04:38.623618  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:38.819918  9198 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 4: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:38.824672  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2144) took 1234 ms. Trace:
I20260709 13:04:38.688601  9189 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:04:38.620786  9087 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.267s (SENT)
W20260709 13:04:38.835690  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 214981us
I20260709 13:04:38.834878  9263 rpcz_store.cc:276] 0709 13:04:37.589861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.219702 (+629841us) service_pool.cc:224] Handling call
0709 13:04:38.219985 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:38.220277 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:38.592989 (+372712us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.823977 (+230988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:38.824028 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:38.824076 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.824361 (+   285us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:38.824469 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10413,"prepare.queue_time_us":21422,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":81,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":145,"thread_start_us":538592,"threads_started":2,"wal-append.queue_time_us":161608}
I20260709 13:04:38.838995  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:38.836479  9087 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.565s (SENT)
I20260709 13:04:38.840015  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:38.600126 12071 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.699824 12175 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:04:38.598337 12122 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:38.840227  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:38.695215 12198 log.cc:927] Time spent T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:04:38.604650 12040 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.692898 12200 raft_consensus.cc:3055] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:38.826012 12134 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:38.823164 12172 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.064s	user 0.011s	sys 0.014s
I20260709 13:04:38.859905  9859 raft_consensus.cc:2368] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5.
I20260709 13:04:38.824226 12064 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:38.747051 11725 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.867966  9861 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:38.894843  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:38.697441 12125 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:38.726816 12074 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:38.600375 11801 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:38.767446 11980 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.929778  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:38.897855  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 61409.3us
I20260709 13:04:38.925822  9856 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:04:38.926810 12156 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:38.964563  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:38.988692  9861 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:38.925151 11970 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:38.969237  9855 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:38.972957  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:38.859964 11974 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:38.975078  9856 pending_rounds.cc:85] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 5
I20260709 13:04:38.997646  9853 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:38.998100  9856 pending_rounds.cc:107] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:04:38.998440  9856 raft_consensus.cc:2889] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:38.896095  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:38.978084  9122 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:38.891291 12167 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:38.980180  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 54569.6us
I20260709 13:04:38.897902 11977 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:38.746032 12023 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:38.994187  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:39.011762  9123 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:38.995995  9858 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:38.897540 12131 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.999835  9268 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:38.979189  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:38.825177 12142 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:39.002928  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 856) took 1259 ms. Trace:
I20260709 13:04:38.898022 12155 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:39.004693  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:39.019348  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:39.020128  9853 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:38.859704 11975 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:38.961810 12064 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:38.924676 12200 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:38.837704 12032 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:38.982112 12187 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:04:39.015314  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 120556us
W20260709 13:04:39.013471 12150 consensus_meta.cc:287] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.583s	user 0.001s	sys 0.009s
I20260709 13:04:38.923972 12071 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.994187 11517 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.019289  9857 rpcz_store.cc:276] 0709 13:04:37.743122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:37.743348 (+   226us) service_pool.cc:224] Handling call
0709 13:04:37.743834 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:37.744236 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:37.744267 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:37.744763 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:04:38.368601 (+623838us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:38.861689 (+493088us) log.cc:844] Serialized 14 byte log entry
0709 13:04:38.861968 (+   279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.862069 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:39.002657 (+140588us) raft_consensus.cc:1685] finished
0709 13:04:39.002684 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:39.002855 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":111122,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":31217,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":53016,"thread_start_us":1101601,"threads_started":2,"wal-append.queue_time_us":700800}
I20260709 13:04:39.028461 12113 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:39.029141 12216 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:39.029251  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:39.032382 12200 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.034965 12114 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:38.994290 12222 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:04:39.036124  9164 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:39.037498 12032 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.022732 12064 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:39.038887 12115 consensus_meta.cc:287] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.750s	user 0.009s	sys 0.000s
I20260709 13:04:39.039537 12158 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.040580 12078 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:38.994624 11980 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.974785 12074 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.040977 12155 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.011198 12122 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.025240 12131 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.042762 12062 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:39.042196 11801 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.019320 11806 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:39.044202 12156 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:39.010262 12167 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.008062 11974 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:38.967800 12040 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.059552  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:39.061939  9196 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:39.098691  9854 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:39.064836  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:39.099555  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:39.026022 12069 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:39.055416 12150 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.068200 12035 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:39.068384 12154 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:39.113456 12154 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.113844 12154 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:39.115011 11974 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:39.073000 11517 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:04:39.073925 11980 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.055855 12221 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.018039 11977 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.109233 12122 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:39.136324 11977 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:39.136476 11517 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:39.137176 12125 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:39.086517  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:39.138062  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:39.087767  9268 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:39.089010 12074 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:39.089957  9271 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:39.092741  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 114191us
I20260709 13:04:39.046509 11975 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:39.096539  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:39.100684  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 261749us
I20260709 13:04:39.040580 12166 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:39.109526  9087 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.730s (SENT)
I20260709 13:04:39.160449  9087 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:39.110191 12177 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:39.123569  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:39.124576  9858 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:39.126027 12150 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:39.126837  9196 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:39.127408 12142 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:39.127730 12114 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:39.128202 11725 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.138340  9272 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:39.068645 12200 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:39.124573 12155 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:39.223690  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 114203us
W20260709 13:04:39.224699  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 81594.8us
I20260709 13:04:39.226225  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:39.183331  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.151877 12035 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:39.222560 12142 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:04:39.229014  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 190431us
W20260709 13:04:39.274408  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 48310.7us
I20260709 13:04:39.275027  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:39.319928 12131 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.320181 12082 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.321400 12167 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:39.322094  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 67647.5us
I20260709 13:04:39.322362 12113 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.323107 12113 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.323760 12125 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:39.326349 12113 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.328171 12156 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:39.328778 12156 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:39.329726 12156 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.234658  9272 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:39.269069  9854 raft_consensus.cc:2410] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:39.272624  9857 raft_consensus.cc:1217] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[3.5-5.6]   Dedup: 3.5->[5.6-5.6]
I20260709 13:04:39.332398  9857 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:04:39.152236 12221 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.338034 12221 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:39.348188  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 238057us
W20260709 13:04:39.349553  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 80981.4us
I20260709 13:04:39.350293  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:39.320927 12082 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:39.363029  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 87887.3us
I20260709 13:04:39.364885  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:39.366346  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:39.368965  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.370436  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:39.376513  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:39.393162  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:39.394333  9269 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
W20260709 13:04:39.314428 12228 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:04:39.377507  9857 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:39.402779  9853 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:04:39.406576  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 96063.5us
I20260709 13:04:39.418936  9269 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:39.432413  9122 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:39.434067  9121 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:39.472627  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.348670 12076 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.502177  9120 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
W20260709 13:04:39.503321  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 76266.7us
I20260709 13:04:39.494120  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:39.399376 12230 raft_consensus.cc:2764] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:04:39.404163 11980 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:39.512440  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:39.530649  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:39.426244 12074 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:39.531790  9121 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 13:04:39.512768  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 75671.3us
I20260709 13:04:39.514053  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:39.425427 12179 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:39.425654 11970 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:39.525285  9115 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:04:39.541237  9119 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:39.533768  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:39.494143 12023 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.432240 11725 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 13:04:39.529731 11980 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.541780 11806 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.505785 12205 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.138s
I20260709 13:04:39.509233 12076 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.571977 12230 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.565253 12231 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:39.405443 12154 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.624352 11725 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:39.641230 11725 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.631361  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:39.643428 11725 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.646191  9119 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:39.536825 12131 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
W20260709 13:04:39.646867  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 416229us
I20260709 13:04:39.542506 12064 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:39.607519 11980 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:39.558487 12074 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.652626  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:39.652804  9121 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:04:39.608105 12234 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:39.630373 12075 consensus_meta.cc:287] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.798s	user 0.009s	sys 0.002s
I20260709 13:04:39.665207  9122 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:39.635501 12205 consensus_queue.cc:1243] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:39.574290 12023 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.639150 12231 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.683384  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:39.685993  9121 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:04:39.701318  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 331324us
I20260709 13:04:39.701251  9193 raft_consensus.cc:606] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:39.702358  9193 raft_consensus.cc:614] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:39.703923  9265 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:39.652263 12229 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:39.706548  9238 leader_election.cc:400] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6.
I20260709 13:04:39.710117  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:39.630597 11806 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.714083  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:39.680655 12234 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.592967 12035 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.727378  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:39.728868  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:39.729427  9272 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:39.730322  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:39.732227  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:39.734849  9269 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:39.736572  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:39.737185  9271 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:39.731536  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:39.703639 12231 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:39.684401 11725 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:04:39.710595 11801 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:04:39.747908  9087 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:39.680176 12075 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.709043 12229 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.715444 11975 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:39.650142 12230 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.726293 12234 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:39.753588  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:04:39.754983  9825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 2.653s (SENT) [suppressed 31 similar messages]
I20260709 13:04:39.760347  9121 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:39.769596  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:39.770241  9121 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 3.
I20260709 13:04:39.773732  9195 raft_consensus.cc:2368] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5.
I20260709 13:04:39.776937  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:39.746676 12082 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:39.751338 12229 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:39.782667  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:39.783428  9115 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:39.752010 12177 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:39.778033  9193 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:39.788579 12244 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:39.754292 12114 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:39.771407 12134 consensus_meta.cc:287] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.782s	user 0.010s	sys 0.001s
I20260709 13:04:39.793658  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:39.751935 12155 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 13:04:39.804898  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 290927us
I20260709 13:04:39.806476  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:39.808909  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:39.809665  9193 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:39.792182 12069 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:39.774081 11975 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:04:39.772919 12172 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:04:39.792469 12216 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
W20260709 13:04:39.828337 12241 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:04:39.828224  9269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:04:39.829177  9269 raft_consensus.cc:606] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:39.829525  9269 raft_consensus.cc:614] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:39.831187 12154 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.802752 12244 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.829205 12156 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:39.833168 12156 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:39.833877 12244 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:39.834077 12218 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 1.047s	user 0.002s	sys 0.019s
I20260709 13:04:39.834869 12156 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:39.835276  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:39.835980  9269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:04:39.836402  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:39.836936 12076 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:39.835378 12156 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:39.803876 12134 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:39.801236 12082 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:04:39.801206 12177 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:39.837399  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:40.127247  9192 raft_consensus.cc:2376] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4. Re-sending same reply.
I20260709 13:04:40.127415  9266 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:40.128067  9269 raft_consensus.cc:606] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:40.128340  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:40.128638  9269 raft_consensus.cc:614] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:40.129621  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.129485  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:40.131119  9192 raft_consensus.cc:2368] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9.
I20260709 13:04:39.804937 12114 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:40.114483  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 325351us
W20260709 13:04:40.131385  9116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.008s	sys 0.012s
I20260709 13:04:39.839713  9115 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:04:39.941601  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 408050us
W20260709 13:04:40.159955  9116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.008s	sys 0.012s
I20260709 13:04:40.161585  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:39.858779  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.094491  9193 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
W20260709 13:04:39.840269  9069 auto_rebalancer.cc:893] Removing replace marker failed: Replica 1efabcb1befc400aadf101e8276fc1aa is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:04:40.170040  9069 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:04:40.177549  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 429708us
I20260709 13:04:40.178576  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:40.178373  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:39.843425 12122 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.708s	user 0.009s	sys 0.001s
W20260709 13:04:39.840768 12239 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:04:40.190065  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2155) took 2018 ms. Trace:
I20260709 13:04:40.190251  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:40.130470 12154 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:40.211335  9198 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:04:40.172205 12156 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:40.145879 12242 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.476s	user 0.001s	sys 0.000s
W20260709 13:04:40.171648  9069 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:04:39.841874 12074 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:40.186347  9118 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:04:39.841543 12117 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:04:39.880070 12238 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:04:40.139582 12023 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:40.213160  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:40.213168  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:40.227025  9115 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:40.214165  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 412676us
I20260709 13:04:40.228991  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:40.177526 12216 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.127918 11874 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:04:40.213064 11725 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:04:40.242573  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 68232us
I20260709 13:04:40.244146  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:40.245151  9267 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:40.190510  9264 rpcz_store.cc:276] 0709 13:04:38.171460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.257408 (+ 85948us) service_pool.cc:224] Handling call
0709 13:04:38.261338 (+  3930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:38.261688 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:38.261715 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:38.262175 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:04:38.408761 (+146586us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:38.408816 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:38.408899 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:38.478539 (+ 69640us) raft_consensus.cc:1685] finished
0709 13:04:38.478568 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.189836 (+1711268us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b54004439d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:40.190004 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":273,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"spinlock_wait_cycles":8252800,"thread_start_us":146188,"threads_started":1,"wal-append.queue_time_us":206465,"wal-append.run_cpu_time_us":1564,"wal-append.run_wall_time_us":1001579}
I20260709 13:04:40.251643  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:40.252723  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:40.253695  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:39.825518 12167 consensus_queue.cc:1059] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.370s
I20260709 13:04:40.222895 12117 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:40.253170  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:40.234872  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 428436us
I20260709 13:04:40.221896 12252 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:40.262790  9115 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:40.303012  9860 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEARNER]: Advancing to term 9
W20260709 13:04:40.247021 12247 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:04:40.257022  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.258711  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:40.233270 12074 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:40.305567  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 75636.7us
I20260709 13:04:40.305675  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:40.307685  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 77766.2us
W20260709 13:04:40.262473 12231 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.006s
I20260709 13:04:40.250411 11801 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:04:40.276108  9264 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:40.308883 12117 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.316372  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2166) took 1230 ms. Trace:
I20260709 13:04:40.313799  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:40.309664  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 116186us
I20260709 13:04:40.261368 12172 consensus_queue.cc:1243] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:40.316855  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:40.317581  9267 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:40.317639  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:40.316747  9270 rpcz_store.cc:276] 0709 13:04:39.085656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.085773 (+   117us) service_pool.cc:224] Handling call
0709 13:04:39.088769 (+  2996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.089146 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.089183 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.089668 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.381416 (+291748us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:04:39.381474 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.381568 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.316197 (+934629us) raft_consensus.cc:1685] finished
0709 13:04:40.316216 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.316318 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":735,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4836,"thread_start_us":251258,"threads_started":1,"wal-append.queue_time_us":484693}
I20260709 13:04:40.314949  9118 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:04:40.318854  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:40.321348 12234 consensus_meta.cc:287] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.551s	user 0.006s	sys 0.003s
I20260709 13:04:40.321789 12253 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:40.327353  9114 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:40.326871  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.330336  9118 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:40.332098  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:40.332317  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 887) took 1058 ms. Trace:
I20260709 13:04:40.332722  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:40.333504  9856 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:40.336632  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.337585  9861 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:40.339123  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.350652  9114 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:40.332831  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:40.349371  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.332701  9858 rpcz_store.cc:276] 0709 13:04:39.284096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.299861 (+ 15765us) service_pool.cc:224] Handling call
0709 13:04:39.300276 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.404429 (+104153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.404479 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.493454 (+ 88975us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.778234 (+284780us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:39.778275 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.778341 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:40.332135 (+553794us) raft_consensus.cc:1685] finished
0709 13:04:40.332152 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:40.332267 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103707,"raft.queue_time_us":124929,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":62074,"thread_start_us":237255,"threads_started":1,"wal-append.queue_time_us":316180}
I20260709 13:04:40.356586  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:40.357259  9858 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:40.239837 12216 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:40.315977 12230 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:40.328670 12155 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:40.326004 11975 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:40.368371  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 55598us
I20260709 13:04:40.384124  9161 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:40.377465  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:40.369364  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:40.378449  9856 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:40.368568  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:40.386831  9270 raft_consensus.cc:2368] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8.
I20260709 13:04:40.386920  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:40.386930  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.387939  9270 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:04:40.388413 12273 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.388545  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.368388 11801 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.388911 12273 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.389492 11801 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:40.391193 12272 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.391446 11801 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:40.367575  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:40.392357  9118 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:40.314010 12074 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.369009  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:40.368772  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 51196.4us
I20260709 13:04:40.387467  9264 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:40.396189  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.401304  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:40.403806  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:40.391881 12272 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.409261  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.409436  9118 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:40.316215 12231 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:40.376042 12117 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:40.429791  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.431613  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.432440  9855 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:40.361843 12134 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:40.382990 11725 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:40.417320  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:40.418092  9089 connection.cc:256] RPC call timeout handler was delayed by -1.6626443110000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
I20260709 13:04:40.391878 12273 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:40.419126 12271 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:40.420109 12252 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:40.421538 12274 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:40.423043  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.402489 12262 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:40.334725 12234 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.397059 12150 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.400126 12245 raft_consensus.cc:3055] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:40.439677 12278 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:40.450426  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:40.418072  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.451872  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.406891 12177 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:40.471419  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.455796  9264 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:40.474743  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.475473  9118 raft_consensus.cc:2393] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:40.458365  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 57144us
I20260709 13:04:40.371057 12154 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:04:40.417742 11801 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:40.460011  9271 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:40.396685 12261 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:40.462342  9116 raft_consensus.cc:2376] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5. Re-sending same reply.
I20260709 13:04:40.367836 12260 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:40.464354  9860 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEARNER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:40.486318  9856 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEARNER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:40.419577 12270 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.422322 12160 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:40.477998  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:40.399991 12155 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.402298 11975 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.453783 12167 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.483042  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.484299  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.460181 12114 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:40.489326  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.471276 12245 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:40.492445  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.460867 11725 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:40.493922  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 43627.4us
I20260709 13:04:40.481871 12262 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:40.494742 12265 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:40.495354 12028 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:40.491287  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.552578  9118 raft_consensus.cc:2410] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:40.501256 12155 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:40.501700 12167 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.554768  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.492362 12260 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:40.466089 12274 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:40.506860  9859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 901) took 2124 ms (client timeout 2069 ms). Trace:
I20260709 13:04:40.512362  9855 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:04:40.503716 11980 consensus_meta.cc:287] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.823s	user 0.004s	sys 0.005s
I20260709 13:04:40.549770  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:40.558574  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 173397us
I20260709 13:04:40.484295 12261 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:40.503328  9861 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:40.479329 12150 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:40.556066  9859 rpcz_store.cc:269] 0709 13:04:38.382909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.383069 (+   160us) service_pool.cc:224] Handling call
0709 13:04:40.506681 (+2123612us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54004a3dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:40.506797 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024692992}
I20260709 13:04:40.519814  9089 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:40.564500  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:40.565949  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46312.4us
I20260709 13:04:40.566466  9118 raft_consensus.cc:2410] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:04:40.564720  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:40.566954  9089 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:40.568835  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:40.570010  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:40.570557  9118 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:40.573339  9114 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:40.501278 11975 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:40.574687  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.550521 12265 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:40.577908  9114 raft_consensus.cc:2410] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:40.548372 11867 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.465188 12271 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.563220 12274 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:40.562400 12261 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:40.566066 12216 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:40.583314  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.496063 12160 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.573146 12028 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:40.586177  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.589253  9859 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:40.564850 11980 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:40.596642  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.602308  9862 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:40.599284  9116 raft_consensus.cc:2410] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:04:40.603075  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:40.493914 12270 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.609858  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:40.585280 12035 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:40.608114  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:40.614813  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:40.616106 12266 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:40.617038 12277 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:40.617506 12259 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.621747  9122 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:04:40.623044 12201 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 2.058s	user 0.006s	sys 0.018s
I20260709 13:04:40.623292 12064 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:40.629936 12142 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:40.586941 11867 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:40.633422  9859 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 6: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:40.653115  9858 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:40.657756  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:40.658138  9272 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:40.658157  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:40.659438  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48213.1us
I20260709 13:04:40.588825 12216 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:40.590938 12272 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:40.661044 11801 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:40.705641  9118 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:40.565032 12252 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.581811 12245 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:40.789676  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:41.065745  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:41.066658  9118 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:41.067051  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 500137us
W20260709 13:04:41.011503  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 50245.5us
I20260709 13:04:41.069133  9089 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:41.070986  9264 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.010s	sys 0.000s
W20260709 13:04:41.071887  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 593959us
W20260709 13:04:41.129640  9264 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.010s	sys 0.000s
W20260709 13:04:40.660463 12200 consensus_meta.cc:287] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.439s	user 0.009s	sys 0.004s
I20260709 13:04:40.831822  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:41.137408  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 305677us
W20260709 13:04:41.137763  9238 connection.cc:256] RPC call timeout handler was delayed by -1.2449056739999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
I20260709 13:04:41.138155  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:41.138214  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:41.138846  9826 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:41.138378  9238 connection.cc:256] RPC call timeout handler was delayed by -1.0019056739999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 13:04:41.139317  9164 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:41.133780  9855 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.126s	sys 0.236s
W20260709 13:04:41.140300  9855 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.126s	sys 0.237s
W20260709 13:04:41.140699  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 66243us
W20260709 13:04:40.906507 12229 consensus_meta.cc:287] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.106s	user 0.011s	sys 0.011s
I20260709 13:04:41.148478  9267 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:41.009341 11817 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:41.063638 12260 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:41.066219 12167 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:40.819890 12231 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:40.851222 12134 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.156442 12200 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:40.689353 12259 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:41.150357  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.141642 12071 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:40.868835 12074 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:40.658987 12035 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.662302 12156 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.159667 11817 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:40.650542 12262 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.973405 12276 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:41.170508  9859 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:40.814817 12216 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.141654 12186 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:41.150171  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:41.145593 12272 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:41.149443 12280 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:04:41.182307 12261 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.588s	user 0.004s	sys 0.004s
W20260709 13:04:41.182717 12268 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.383s	user 0.000s	sys 0.001s
I20260709 13:04:41.183535 12271 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.183213 12134 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.183939 12285 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:41.184725 12270 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.177062  9189 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:41.071867 12274 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:41.189261 12283 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:04:41.198330  9118 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:41.130439 12279 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.176781 12221 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:40.827464 12028 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.172415  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:41.191152 11980 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:41.161680 12229 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.230414  9114 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:41.224514 12276 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:41.238132  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 93598.5us
I20260709 13:04:41.277984  9826 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:41.245220  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:41.279163  9122 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:41.280393  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:41.187966 12200 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.208321 12074 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:04:40.830408 11801 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:40.854316 12265 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:41.232455 12235 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 1.667s	user 0.004s	sys 0.016s
I20260709 13:04:41.257648  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:41.198441  9192 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.014s	sys 0.019s
I20260709 13:04:41.260283  9862 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.9-6.9]
W20260709 13:04:41.284904  9192 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.015s	sys 0.019s
W20260709 13:04:41.263291  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 127534us
I20260709 13:04:41.261060  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:41.273406  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:41.286254  9118 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
W20260709 13:04:41.275905  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 135153us
I20260709 13:04:41.287238  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:41.276531  9859 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:41.292061  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 664) took 1105 ms. Trace:
I20260709 13:04:41.292385 11980 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:41.294454  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:41.295527 11980 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:41.295549 12280 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:41.302008  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.302508  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:41.293196  9264 rpcz_store.cc:276] 0709 13:04:40.186449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.475194 (+288745us) service_pool.cc:224] Handling call
0709 13:04:40.475443 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.475649 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:41.130080 (+654431us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.291624 (+161544us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:41.291687 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:41.291775 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.291860 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.291993 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":707029,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":342,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":809144,"threads_started":2}
I20260709 13:04:41.286033 12160 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:41.247990 12261 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.311990  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.308416  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:41.317917  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:41.318970  9862 raft_consensus.cc:2368] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5.
I20260709 13:04:41.318950  9238 leader_election.cc:400] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9.
I20260709 13:04:41.320796  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:41.320400  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:41.328066  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.329044  9118 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:41.301918 12295 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:41.331192  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:04:41.323246 12286 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:41.333096  9089 leader_election.cc:400] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5.
I20260709 13:04:41.335282  9089 leader_election.cc:403] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:04:41.267999 12279 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.338004 12279 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.340164  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:41.324224 12216 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.289244 12270 consensus_queue.cc:1059] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:04:41.260061 12285 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:41.312855 12229 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:41.291044 12221 consensus_queue.cc:1175] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "92919b70789041c8a0de0b3525154fd1" responder_term: 5 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:04:41.330685  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.259377 11817 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.333700  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:41.348871  9189 raft_consensus.cc:2368] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7.
I20260709 13:04:41.340893  9862 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:41.309274 12280 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.314572 12271 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:04:41.350888  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:41.340929 12230 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:41.311360 12300 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:41.314646 12296 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:04:41.357898  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:41.355204 12294 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:41.353881 12291 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:41.355916  9118 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:41.358173 12156 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.347493 12295 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:41.351979  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:41.360368  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.362833  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:41.362977  9862 raft_consensus.cc:2410] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:41.363030  9192 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:41.353034  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.366786  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.367596  9192 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:41.356027 12131 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:41.355149 12167 raft_consensus.cc:3055] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:04:41.374744  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.377127  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:41.377574  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.361500 12296 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:41.356748 12231 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.355815 12035 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:41.370468  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.387887  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.390060  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.361603 12230 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:41.381567 12291 raft_consensus.cc:697] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:04:41.449710  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 100054us
I20260709 13:04:41.425419  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.450606  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:41.407963  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:41.402884  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:41.452000  9192 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:41.453182 12291 consensus_queue.cc:237] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:41.454347  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:41.454517 12271 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:41.386379 12131 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:41.456359  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:41.403838 12134 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.393208 12276 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.361796 12259 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:41.404351 12231 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.460738  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2013) took 2032 ms. Trace:
I20260709 13:04:41.466884 12230 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.460845 12131 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:41.469965 12285 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:41.468447  9123 rpcz_store.cc:276] 0709 13:04:39.428742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.431832 (+  3090us) service_pool.cc:224] Handling call
0709 13:04:39.493598 (+ 61766us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:39.528479 (+ 34881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.528523 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:40.586851 (+1058328us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b2c00000f48. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:40.604182 (+ 17331us) log.cc:844] Serialized 52 byte log entry
0709 13:04:40.801158 (+196976us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:40.801899 (+   741us) log.cc:844] Serialized 14 byte log entry
0709 13:04:40.802122 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:40.802216 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.456488 (+654272us) raft_consensus.cc:1685] finished
0709 13:04:41.460127 (+  3639us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:41.460679 (+   552us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9284,"prepare.queue_time_us":237,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":46871,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":69459,"spinlock_wait_cycles":37622656,"thread_start_us":168920,"threads_started":1,"wal-append.queue_time_us":560760}
I20260709 13:04:41.468758  9089 leader_election.cc:400] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7.
I20260709 13:04:41.473282  9089 leader_election.cc:403] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:04:41.384657 12167 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:41.466151 12277 consensus_meta.cc:287] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.804s	user 0.007s	sys 0.003s
I20260709 13:04:41.389895 12296 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:41.425206 12261 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.474702  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:41.428252 12274 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:41.458669 12262 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.483270 12234 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.484365  9264 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:41.484913  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:41.472668 12269 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:41.393890 12297 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5.
I20260709 13:04:41.488976  9192 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:41.393863 12295 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.473973 11980 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:41.838320  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.497795  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:42.085587  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:41.515609 12256 raft_consensus.cc:3055] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:42.086237 12256 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:42.086374  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:42.086716  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:41.515936 12131 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:42.087124 12256 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:41.524572 11806 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:42.085180 12231 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:41.870595 12167 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:41.511518  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:41.480360 12277 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:42.169528 12167 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:41.546666 12179 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:42.170225  9268 raft_consensus.cc:2410] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:04:41.489872 12262 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:42.084429 12269 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:42.171653  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.173100  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.597302  9268 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:42.598338  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 671) took 1231 ms. Trace:
I20260709 13:04:42.598784  9268 rpcz_store.cc:276] 0709 13:04:41.366647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.172771 (+806124us) service_pool.cc:224] Handling call
0709 13:04:42.598148 (+425377us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:42.491093  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:42.495234  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:41.548650  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 499) took 1228 ms. Trace:
I20260709 13:04:41.517897  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:42.500373  9118 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:41.880223  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:42.070431  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.087662  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:42.146971  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:42.071235 11725 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:04:41.791904 12023 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:42.667804 12023 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:42.022811 12040 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:41.496353 12117 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.047s	user 0.013s	sys 0.000s
I20260709 13:04:42.247670 12131 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:42.083380 12150 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:42.170514 12179 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:42.313051 12256 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:42.311234 12262 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:41.489470 12270 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:42.719834 12179 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:42.721661 12179 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:42.595901  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:42.612350 12311 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.106s	user 0.013s	sys 0.025s
I20260709 13:04:42.632606  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 2040) took 1281 ms. Trace:
I20260709 13:04:42.724383  9117 rpcz_store.cc:276] 0709 13:04:41.351181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.517530 (+166349us) service_pool.cc:224] Handling call
0709 13:04:42.632350 (+1114820us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.726673  9117 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:04:42.727015  9117 pending_rounds.cc:85] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 5
I20260709 13:04:42.727236  9117 pending_rounds.cc:107] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:04:42.727485  9117 raft_consensus.cc:2889] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.646932  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1748) took 1292 ms. Trace:
I20260709 13:04:42.727887  9264 rpcz_store.cc:276] 0709 13:04:41.354630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.355103 (+   473us) service_pool.cc:224] Handling call
0709 13:04:42.646475 (+1291372us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54003a25d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:42.646716 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288814976}
I20260709 13:04:42.729398  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:42.648458  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 663) took 2462 ms (client timeout 2231 ms). Trace:
I20260709 13:04:42.649886  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:42.730229  9267 rpcz_store.cc:269] 0709 13:04:40.186170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.326631 (+140461us) service_pool.cc:224] Handling call
0709 13:04:42.647930 (+2321299us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400062054. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:42.648186 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1753638144}
I20260709 13:04:42.730942  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1943) took 2025 ms. Trace:
I20260709 13:04:42.650770  9114 rpcz_store.cc:276] 0709 13:04:40.320254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.602782 (+282528us) service_pool.cc:224] Handling call
0709 13:04:41.548338 (+945556us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b5400254c54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:41.548563 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1434848640}
I20260709 13:04:42.731477  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.732666  9267 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:42.733209  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 670) took 1366 ms. Trace:
I20260709 13:04:42.733736  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 503) took 2412 ms. Trace:
I20260709 13:04:42.652405  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.734575  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 518) took 1449 ms. Trace:
I20260709 13:04:42.735101  9123 rpcz_store.cc:276] 0709 13:04:41.258217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.879848 (+621631us) service_pool.cc:224] Handling call
0709 13:04:42.705742 (+825894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.735178  9268 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:42.666298  9190 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:42.647360  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:42.699810  9194 raft_consensus.cc:2410] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:04:42.736407  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.734355  9114 rpcz_store.cc:276] 0709 13:04:40.321228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.732066 (+2410838us) service_pool.cc:224] Handling call
0709 13:04:42.732620 (+   554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.733230 (+   610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.733269 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.733366 (+    97us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:42.733439 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.733512 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.733651 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.731487  9191 rpcz_store.cc:276] 0709 13:04:40.705774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.648933 (+1943159us) service_pool.cc:224] Handling call
0709 13:04:42.730827 (+ 81894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.733533  9267 rpcz_store.cc:276] 0709 13:04:41.366418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.731134 (+1364716us) service_pool.cc:224] Handling call
0709 13:04:42.733138 (+  2004us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:42.736260  9268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 668) took 1370 ms (client timeout 1716 ms). Trace:
W20260709 13:04:42.754765  9268 rpcz_store.cc:269] 0709 13:04:41.365982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.636924 (+1270942us) service_pool.cc:224] Handling call
0709 13:04:42.736127 (+ 99203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.760040  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.767082  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1954) took 1616 ms. Trace:
I20260709 13:04:42.772285  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1749) took 1413 ms. Trace:
I20260709 13:04:42.774974  9268 rpcz_store.cc:276] 0709 13:04:41.356166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.759550 (+1403384us) service_pool.cc:224] Handling call
0709 13:04:42.769396 (+  9846us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.781407  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:42.782819  9191 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:42.783365  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 531) took 2174 ms. Trace:
W20260709 13:04:42.783954 12307 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.237s	user 0.011s	sys 0.028s
I20260709 13:04:42.785904  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2035) took 2203 ms. Trace:
I20260709 13:04:42.786686  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 665) took 2599 ms. Trace:
I20260709 13:04:42.787577  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.788470  9122 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:04:42.788933  9122 pending_rounds.cc:85] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 7
I20260709 13:04:42.789228  9122 pending_rounds.cc:107] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 6.8
W20260709 13:04:42.789287 12251 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 2.392s	user 0.016s	sys 0.036s
I20260709 13:04:42.789573  9122 raft_consensus.cc:2889] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:42.787168  9270 rpcz_store.cc:276] 0709 13:04:40.186884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.084460 (+897576us) service_pool.cc:224] Handling call
0709 13:04:41.129575 (+ 45115us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:41.289713 (+160138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:41.289744 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:41.304269 (+ 14525us) log.cc:844] Serialized 52 byte log entry
0709 13:04:41.386572 (+ 82303us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:41.456283 (+ 69711us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.456847 (+   564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:41.456966 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.786107 (+1329141us) raft_consensus.cc:1685] finished
0709 13:04:42.786147 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.786404 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1537,"prepare.queue_time_us":15517,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":38998,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":19013,"thread_start_us":203338,"threads_started":2,"wal-append.queue_time_us":116915}
W20260709 13:04:42.790179 12250 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 2.858s	user 0.009s	sys 0.016s
I20260709 13:04:42.790882  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.791126 12297 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:42.791767  9270 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:42.792090 12297 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:42.792922  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.793727  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 669) took 1427 ms. Trace:
I20260709 13:04:42.794113  9270 rpcz_store.cc:276] 0709 13:04:41.366211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.792627 (+1426416us) service_pool.cc:224] Handling call
0709 13:04:42.793643 (+  1016us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.795073  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:42.788100  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1947) took 1411 ms. Trace:
I20260709 13:04:42.773422  9190 rpcz_store.cc:276] 0709 13:04:41.150449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.087424 (+936975us) service_pool.cc:224] Handling call
0709 13:04:42.762570 (+675146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25344}
I20260709 13:04:42.786562  9114 rpcz_store.cc:276] 0709 13:04:40.582490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.784416 (+2201926us) service_pool.cc:224] Handling call
0709 13:04:42.784934 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.785363 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.785408 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.785492 (+    84us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:42.785586 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.785669 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.785821 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.796265  9191 rpcz_store.cc:276] 0709 13:04:40.609090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.777829 (+2168739us) service_pool.cc:224] Handling call
0709 13:04:42.783272 (+  5443us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.789237  9267 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:42.787768  9263 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:42.812583  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.790525  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 497) took 2470 ms. Trace:
I20260709 13:04:42.813346  9122 rpcz_store.cc:276] 0709 13:04:40.319726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.634707 (+2314981us) service_pool.cc:224] Handling call
0709 13:04:42.635523 (+   816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.790271 (+154748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.790455 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.813120  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 502) took 2489 ms. Trace:
I20260709 13:04:42.816283  9117 rpcz_store.cc:276] 0709 13:04:40.320991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.725717 (+2404726us) service_pool.cc:224] Handling call
0709 13:04:42.726117 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.808966 (+ 82849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.810273 (+  1307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.814630  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 672) took 1444 ms. Trace:
I20260709 13:04:42.787530 12227 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.818701  9267 rpcz_store.cc:276] 0709 13:04:41.368188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.756644 (+1388456us) service_pool.cc:224] Handling call
0709 13:04:42.812162 (+ 55518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.818902  9123 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:42.820894  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 505) took 2499 ms. Trace:
I20260709 13:04:42.821326  9123 rpcz_store.cc:276] 0709 13:04:40.321711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.736169 (+2414458us) service_pool.cc:224] Handling call
0709 13:04:42.778861 (+ 42692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.820597 (+ 41736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.820831 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.813380  9192 rpcz_store.cc:276] 0709 13:04:41.353001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.087438 (+734437us) service_pool.cc:224] Handling call
0709 13:04:42.760633 (+673195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.822866 12227 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:42.830634  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.832875  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:42.834383  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 526) took 2244 ms. Trace:
I20260709 13:04:42.834962  9191 rpcz_store.cc:276] 0709 13:04:40.590408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.828586 (+2238178us) service_pool.cc:224] Handling call
0709 13:04:42.834288 (+  5702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.838716  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:42.836169 12230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.209s	user 0.006s	sys 0.024s
I20260709 13:04:42.849265  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:42.853091 12280 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.357s	user 0.059s	sys 0.191s
W20260709 13:04:42.853588 12280 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.391s	user 0.060s	sys 0.192s
I20260709 13:04:42.855243  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2110) took 2155 ms. Trace:
I20260709 13:04:42.855732  9117 rpcz_store.cc:276] 0709 13:04:40.699901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.826513 (+2126612us) service_pool.cc:224] Handling call
0709 13:04:42.844862 (+ 18349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.854799 (+  9937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.854833 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.854903 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:42.854935 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.855023 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.855158 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:42.850750 12230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.349s	user 0.006s	sys 0.025s
I20260709 13:04:42.858911 12249 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.849761 12023 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:42.852600  9121 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.198s	user 0.008s	sys 0.011s
W20260709 13:04:42.864264  9121 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.304s	user 0.009s	sys 0.011s
I20260709 13:04:42.866298  9190 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:42.876114  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:42.876986  9116 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 516) took 2539 ms (client timeout 2653 ms). Trace:
W20260709 13:04:42.877676  9116 rpcz_store.cc:269] 0709 13:04:40.337219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.482748 (+145529us) service_pool.cc:224] Handling call
0709 13:04:42.876405 (+2393657us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400254254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:42.876775 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244379008}
I20260709 13:04:42.843066 12297 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:42.881142  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:42.882378  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1959) took 1697 ms. Trace:
I20260709 13:04:42.882856  9197 rpcz_store.cc:276] 0709 13:04:41.185307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.880510 (+1695203us) service_pool.cc:224] Handling call
0709 13:04:42.882282 (+  1772us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.852272  9268 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:42.885080  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.886381  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:42.892549  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:42.877107  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1948) took 1523 ms. Trace:
I20260709 13:04:42.895136  9192 rpcz_store.cc:276] 0709 13:04:41.353974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.839586 (+1485612us) service_pool.cc:224] Handling call
0709 13:04:42.877016 (+ 37430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.887017  9268 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:42.895581  9118 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:42.899073  9116 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:42.846289  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.905452  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2039) took 1894 ms. Trace:
I20260709 13:04:42.905875  9116 rpcz_store.cc:276] 0709 13:04:41.008141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.878860 (+1870719us) service_pool.cc:224] Handling call
0709 13:04:42.879387 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.900162 (+ 20775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.902717 (+  2555us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.908587  9191 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:42.895713 12269 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:42.907534  9270 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:42.928438  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2105) took 2744 ms. Trace:
I20260709 13:04:42.969223  9190 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:42.964300  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2024) took 3295 ms. Trace:
I20260709 13:04:42.988595  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 529) took 2374 ms. Trace:
I20260709 13:04:42.989032  9190 rpcz_store.cc:276] 0709 13:04:40.596262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.826756 (+2230494us) service_pool.cc:224] Handling call
0709 13:04:42.970128 (+143372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.896210 12249 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:42.991320  9197 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:42.947599  9121 rpcz_store.cc:276] 0709 13:04:40.183869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.184639 (+   770us) service_pool.cc:224] Handling call
0709 13:04:40.193186 (+  8547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:40.193749 (+   563us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:40.226223 (+ 32474us) log.cc:844] Serialized 14 byte log entry
0709 13:04:41.559908 (+1333685us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b1800085c00. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:04:42.864836 (+1304928us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.865595 (+   759us) log.cc:844] Serialized 26 byte log entry
0709 13:04:42.865756 (+   161us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:42.865794 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.865905 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:42.899281 (+ 33376us) raft_consensus.cc:1685] finished
0709 13:04:42.905128 (+  5847us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.928350 (+ 23222us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12605,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":4357,"raft.queue_time_us":83874,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":11314,"spinlock_wait_cycles":25094912,"thread_start_us":1304556,"threads_started":1,"wal-append.queue_time_us":2620861}
I20260709 13:04:42.994338  9263 raft_consensus.cc:2410] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:04:42.995522  9263 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1750) took 1639 ms (client timeout 1804 ms). Trace:
I20260709 13:04:42.995193  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2106) took 2340 ms. Trace:
I20260709 13:04:42.995841  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:42.996685  9114 rpcz_store.cc:276] 0709 13:04:40.654613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.813908 (+2159295us) service_pool.cc:224] Handling call
0709 13:04:42.907714 (+ 93806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.992875 (+ 85161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.992907 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.994974 (+  2067us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:42.995009 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.995062 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.995149 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:42.997916  9123 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:42.999495  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 673) took 1631 ms. Trace:
W20260709 13:04:42.999835  9826 connection.cc:256] RPC call timeout handler was delayed by -1.1295770770000217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
I20260709 13:04:42.999975 12289 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:43.000694  9826 connection.cc:256] RPC call timeout handler was delayed by -1.0492413539999461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2231s for a response
W20260709 13:04:43.001214  9826 connection.cc:256] RPC call timeout handler was delayed by -1.0438303689999771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
I20260709 13:04:43.001014 12289 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.003180 12150 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:43.004062 12150 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.006788 12131 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.998978  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2037) took 2072 ms. Trace:
I20260709 13:04:42.999976  9267 rpcz_store.cc:276] 0709 13:04:41.368414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.951475 (+1583061us) service_pool.cc:224] Handling call
0709 13:04:42.999432 (+ 47957us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.009650  9123 rpcz_store.cc:276] 0709 13:04:40.926167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.905610 (+1979443us) service_pool.cc:224] Handling call
0709 13:04:42.965658 (+ 60048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.998370 (+ 32712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.998929 (+   559us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:42.997177  9263 rpcz_store.cc:269] 0709 13:04:41.362435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.646623 (+1284188us) service_pool.cc:224] Handling call
0709 13:04:42.995457 (+348834us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.007517 12150 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.024187  9267 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:42.988857  9120 rpcz_store.cc:276] 0709 13:04:39.535807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.535974 (+   167us) service_pool.cc:224] Handling call
0709 13:04:39.536343 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:39.560112 (+ 23769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:39.560182 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:39.571463 (+ 11281us) log.cc:844] Serialized 26 byte log entry
0709 13:04:39.802613 (+231150us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:39.802652 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:39.802715 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:41.508389 (+1705674us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b2400168cf8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765347d9c4 00007f765347bacb 00007f7653636dc9 00007f7653636a55 00007f765363609d 00007f765626810e 00007f765627f88c 00007f765627ebb6 00007f765627e7d8 00007f7656288221
0709 13:04:42.811860 (+1303471us) raft_consensus.cc:1685] finished
0709 13:04:42.814348 (+  2488us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:42.831243 (+ 16895us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24063,"raft.queue_time_us":23948,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":9543,"spinlock_wait_cycles":24953088,"thread_start_us":146829,"threads_started":1,"wal-append.queue_time_us":208197}
I20260709 13:04:43.025424 12221 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:43.030062 12249 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.029727 12221 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:43.019846  9235 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.021s	user 0.018s	sys 0.043s
W20260709 13:04:43.044025 12295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.026s	sys 0.039s
W20260709 13:04:43.044847 12295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.028s	sys 0.040s
I20260709 13:04:43.045354  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2180) took 1508 ms. Trace:
W20260709 13:04:43.027637  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.075s	sys 0.164s
I20260709 13:04:43.047439  9267 rpcz_store.cc:276] 0709 13:04:41.517016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.012017 (+1495001us) service_pool.cc:224] Handling call
0709 13:04:43.016983 (+  4966us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.025103 (+  8120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.025335 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:43.047653  9235 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.149s	user 0.018s	sys 0.043s
W20260709 13:04:43.060552  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.076s	sys 0.164s
I20260709 13:04:43.007637 12131 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:43.046088  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 412915us
I20260709 13:04:43.067428  9270 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:04:43.025234 12291 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.029s	sys 0.036s
I20260709 13:04:43.082952  9190 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:43.088977  9268 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:43.089941  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2186) took 1011 ms. Trace:
I20260709 13:04:43.090392  9268 rpcz_store.cc:276] 0709 13:04:42.078841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.886118 (+807277us) service_pool.cc:224] Handling call
0709 13:04:43.089866 (+203748us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.085050  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2178) took 1563 ms. Trace:
I20260709 13:04:43.091910  9270 rpcz_store.cc:276] 0709 13:04:41.516371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.991026 (+1474655us) service_pool.cc:224] Handling call
0709 13:04:43.046597 (+ 55571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.079182 (+ 32585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.079509 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.091840  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1944) took 2296 ms. Trace:
I20260709 13:04:43.095003  9190 rpcz_store.cc:276] 0709 13:04:40.791155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.991881 (+2200726us) service_pool.cc:224] Handling call
0709 13:04:42.997418 (+  5537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.087186 (+ 89768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.087422 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.098356  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2185) took 1018 ms. Trace:
W20260709 13:04:43.094318 12291 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.068s	user 0.029s	sys 0.036s
I20260709 13:04:43.099821  9267 rpcz_store.cc:276] 0709 13:04:42.078104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.048935 (+970831us) service_pool.cc:224] Handling call
0709 13:04:43.086350 (+ 37415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.093450 (+  7100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:43.093493 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:43.096244 (+  2751us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:43.096303 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.096366 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.096483 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.104655  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2182) took 1028 ms. Trace:
I20260709 13:04:43.104851 12320 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:43.105341  9263 rpcz_store.cc:276] 0709 13:04:42.076029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.017311 (+941282us) service_pool.cc:224] Handling call
0709 13:04:43.065057 (+ 47746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.104123 (+ 39066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:43.104178 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:43.104271 (+    93us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:43.104333 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.104422 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.104579 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.105851 12320 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:43.107515  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 521) took 1848 ms. Trace:
I20260709 13:04:43.107800  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56116 (request call id 20) took 3178 ms. Trace:
I20260709 13:04:43.108246  9269 rpcz_store.cc:276] 0709 13:04:39.832659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.835833 (+  3174us) service_pool.cc:224] Handling call
0709 13:04:42.992908 (+3157075us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b5400303ed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566de853 00007f765b6ac60a 00007f7654c327d4 00007f7654c32762 00007f7654c32691 00007f7654c3260c 00007f7654c318c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.010814 (+ 17906us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670263680}
I20260709 13:04:43.108072  9115 rpcz_store.cc:276] 0709 13:04:41.259164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.885014 (+1625850us) service_pool.cc:224] Handling call
0709 13:04:42.885476 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.886215 (+   739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.886253 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.886840 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.083824 (+196984us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:43.083879 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.084001 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.107059 (+ 23058us) raft_consensus.cc:1685] finished
0709 13:04:43.107086 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.107303 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":818,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"thread_start_us":190878,"threads_started":1,"wal-append.queue_time_us":215651}
I20260709 13:04:43.114360  9191 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:04:43.116564 12221 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.117980  9191 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:04:43.119390  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1957) took 1949 ms. Trace:
W20260709 13:04:43.120069  9162 connection.cc:256] RPC call timeout handler was delayed by -1.2778005839999764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 13:04:43.120613  9162 connection.cc:256] RPC call timeout handler was delayed by -1.0629024020000202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
I20260709 13:04:43.120091  9191 rpcz_store.cc:276] 0709 13:04:41.170843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.116272 (+1945429us) service_pool.cc:224] Handling call
0709 13:04:43.117149 (+   877us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.118841 (+  1692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.119258 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.093657 12249 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:43.124342 12249 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:43.125351 12131 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.126667 12249 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.127942 12289 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:43.128763  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 1.23091e+06us
I20260709 13:04:43.129436 12069 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:43.129791  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.130146 12069 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:43.129989  9269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:04:43.130968  9269 raft_consensus.cc:606] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:43.131356  9269 raft_consensus.cc:614] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:43.127988  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:43.138131  9196 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1944) took 4078 ms (client timeout 1873 ms). Trace:
W20260709 13:04:43.138655  9196 rpcz_store.cc:269] 0709 13:04:39.059620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.099312 (+ 39692us) service_pool.cc:224] Handling call
0709 13:04:43.137678 (+4038366us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b54000a7654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.137884 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61096832}
I20260709 13:04:43.145432  9190 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:43.148734  9191 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:43.149870  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1945) took 1928 ms. Trace:
I20260709 13:04:43.150385  9191 rpcz_store.cc:276] 0709 13:04:41.221779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.141797 (+1920018us) service_pool.cc:224] Handling call
0709 13:04:43.148330 (+  6533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.149487 (+  1157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.149757 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.151762  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1949) took 1791 ms. Trace:
W20260709 13:04:43.157812  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.121206 12280 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.157975 12125 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:43.158308  9190 rpcz_store.cc:276] 0709 13:04:41.354366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.130803 (+1776437us) service_pool.cc:224] Handling call
0709 13:04:43.144970 (+ 14167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:43.145860 (+   890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.146019 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:43.152361 12179 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.159049 12280 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.159684 12179 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:43.162196  9087 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.804s (SENT)
I20260709 13:04:43.162274 12179 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:43.152379  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.162880  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:43.153479 12270 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7.
I20260709 13:04:43.157409 12108 consensus_queue.cc:1059] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:04:43.164019  9235 connection.cc:256] RPC call timeout handler was delayed by -1.0327638209999463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
I20260709 13:04:43.152145 12291 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:43.164618 12303 raft_consensus.cc:3055] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:43.165135 12303 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:43.165006 12291 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.166461 12303 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:43.167080  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:43.167820 12303 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:43.168164 12291 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:43.162712 12280 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:43.168330  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:43.169576  9235 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:43.169691  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 422897us
I20260709 13:04:43.169962 12323 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:43.170281 12179 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:43.174527 12179 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:43.163872 12256 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:43.176195 12256 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:43.176344 12320 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.175861 12179 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:43.176633 12256 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:43.183679  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 103160us
I20260709 13:04:43.185665  9269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
W20260709 13:04:43.174535  9087 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.250s (SENT)
I20260709 13:04:43.189563  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1938) took 2479 ms. Trace:
I20260709 13:04:43.193302  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:43.188763  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:43.188788  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.185443 12291 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:43.192095  9192 rpcz_store.cc:276] 0709 13:04:40.704319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.896583 (+2192264us) service_pool.cc:224] Handling call
0709 13:04:42.897124 (+   541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.898173 (+  1049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.898197 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:42.898840 (+   643us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.095983 (+197143us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:43.096033 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.096684 (+   651us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.179004 (+ 82320us) raft_consensus.cc:1685] finished
0709 13:04:43.179032 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.183285 (+  4253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":148,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":191,"thread_start_us":141552,"threads_started":1,"wal-append.queue_time_us":264327}
W20260709 13:04:43.202579  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.207267 12115 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:43.207856 12317 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:43.207933 12115 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:43.208621 12317 raft_consensus.cc:697] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:43.209226 12115 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:43.209765 12317 consensus_queue.cc:237] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:43.206128 12320 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:43.212965  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1745) took 3484 ms. Trace:
I20260709 13:04:43.213352  9271 rpcz_store.cc:276] 0709 13:04:39.728160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.740868 (+ 12708us) service_pool.cc:224] Handling call
0709 13:04:39.750189 (+  9321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.482146 (+731957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.212498 (+2730352us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b540028d7d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:43.212773 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632058240}
I20260709 13:04:43.216534  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.218132  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:43.218327  9825 connection.cc:256] RPC call timeout handler was delayed by -2.4826826260000416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 13:04:43.218786  9825 connection.cc:256] RPC call timeout handler was delayed by -2.3820073790000151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2745s for a response
W20260709 13:04:43.219112  9825 connection.cc:256] RPC call timeout handler was delayed by -2.3601883289999819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
W20260709 13:04:43.219439  9825 connection.cc:256] RPC call timeout handler was delayed by -2.2953748260000566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 13:04:43.219903 12320 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.220023  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.219254  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.220626 12320 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:43.223285 12320 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.225437  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.225745  9115 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:43.226166  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:43.226722  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.226869  9859 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:43.201560 12291 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.232457 12291 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:43.241014  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72855.7us
W20260709 13:04:43.241662  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.235124 12069 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.243980  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.247092  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.247942 12069 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.256642  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.257541  9192 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
W20260709 13:04:43.248086  9854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 874) took 4271 ms (client timeout 2230 ms). Trace:
I20260709 13:04:43.248310  9857 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:43.254673  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 872) took 2913 ms. Trace:
W20260709 13:04:43.260938  9825 connection.cc:256] RPC call timeout handler was delayed by -2.1990185620000489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 13:04:43.245206  9826 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.706s (SENT)
W20260709 13:04:43.261615  9825 connection.cc:256] RPC call timeout handler was delayed by -2.1738185620000081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
W20260709 13:04:43.264295  9825 connection.cc:256] RPC call timeout handler was delayed by -2.15851856200004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:04:43.264756  9825 connection.cc:256] RPC call timeout handler was delayed by -2.1300073790000056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3025s for a response
W20260709 13:04:43.265231  9825 connection.cc:256] RPC call timeout handler was delayed by -1.8145437470000161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25320000000000004s for a response
W20260709 13:04:43.265691  9825 connection.cc:256] RPC call timeout handler was delayed by -1.3904109130000393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2231s for a response
W20260709 13:04:43.266142  9825 connection.cc:256] RPC call timeout handler was delayed by -1.2786309130000291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 13:04:43.259178  9854 rpcz_store.cc:269] 0709 13:04:38.976195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.978885 (+  2690us) service_pool.cc:224] Handling call
0709 13:04:43.247547 (+4268662us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b540064ffd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.247820 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185763072}
W20260709 13:04:43.267568  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63765.8us
W20260709 13:04:43.254693  9198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1943) took 4195 ms (client timeout 2143 ms). Trace:
I20260709 13:04:43.259780  9856 rpcz_store.cc:276] 0709 13:04:40.341425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.428121 (+ 86696us) service_pool.cc:224] Handling call
0709 13:04:40.486054 (+ 57933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.486911 (+   857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.254082 (+2767171us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b540070f150. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.254371 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132426112}
W20260709 13:04:43.269289  9198 rpcz_store.cc:269] 0709 13:04:39.059328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.137732 (+ 78404us) service_pool.cc:224] Handling call
0709 13:04:43.254227 (+4116495us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540010c054. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.254467 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2430344448}
W20260709 13:04:43.271567  9825 connection.cc:256] RPC call timeout handler was delayed by -1.2378883289999862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33330000000000004s for a response
W20260709 13:04:43.273646  9825 connection.cc:256] RPC call timeout handler was delayed by -1.167221138000059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
I20260709 13:04:43.275370  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:43.275610  9825 connection.cc:256] RPC call timeout handler was delayed by -1.1648634040000729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 13:04:43.276420  9825 connection.cc:256] RPC call timeout handler was delayed by -1.1118204420000666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3305s for a response
W20260709 13:04:43.281838  9825 connection.cc:256] RPC call timeout handler was delayed by -1.0106109130000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26530000000000004s for a response
I20260709 13:04:43.288280 12266 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.289040 12266 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:43.291316 12266 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.292402  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.294112  9190 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:43.295320 12227 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:43.296382  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2038) took 2330 ms. Trace:
I20260709 13:04:43.297142 12069 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.297963 12069 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.298870  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:43.299634  9856 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:43.300498 12292 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 2.026s	user 0.028s	sys 0.006s
I20260709 13:04:43.300572 12069 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:43.297415  9828 connection.cc:256] RPC call timeout handler was delayed by -1.399684322999974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
W20260709 13:04:43.301396  9828 connection.cc:256] RPC call timeout handler was delayed by -1.29168432299997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 13:04:43.301775  9828 connection.cc:256] RPC call timeout handler was delayed by -1.1394156849999604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1947s for a response
W20260709 13:04:43.302117  9828 connection.cc:256] RPC call timeout handler was delayed by -1.1062843229999544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
W20260709 13:04:43.302428  9828 connection.cc:256] RPC call timeout handler was delayed by -1.0827156849999255s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
W20260709 13:04:43.302781  9828 connection.cc:256] RPC call timeout handler was delayed by -1.0477843229999735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 13:04:43.304890  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:43.305840  9854 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.9-6.9]
I20260709 13:04:43.306559  9828 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:43.272444 12200 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:43.296814  9117 rpcz_store.cc:276] 0709 13:04:40.966530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.856925 (+1890395us) service_pool.cc:224] Handling call
0709 13:04:42.857410 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:42.899322 (+ 41912us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:43.093628 (+194306us) log.cc:844] Serialized 14 byte log entry
0709 13:04:43.205655 (+112027us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:43.244786 (+ 39131us) log.cc:844] Serialized 26 byte log entry
0709 13:04:43.244946 (+   160us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:43.244992 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.245073 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.296146 (+ 51073us) raft_consensus.cc:1685] finished
0709 13:04:43.296168 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.296301 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":192657,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":974,"raft.queue_time_us":22354,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":29234,"thread_start_us":179324,"threads_started":2,"wal-append.queue_time_us":129575}
I20260709 13:04:43.319993  9859 raft_consensus.cc:2410] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:43.322679  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:43.329382  9854 raft_consensus.cc:2410] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:04:43.332105  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 89138.8us
I20260709 13:04:43.333753  9859 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:43.335489  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:43.338009  9860 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:43.333847  9826 leader_election.cc:336] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.716s (SENT)
W20260709 13:04:43.340006  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 165565us
W20260709 13:04:43.340852  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.341929  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:43.342496  9190 raft_consensus.cc:2376] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10. Re-sending same reply.
I20260709 13:04:43.345534 12150 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:43.356637  9856 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:43.359496  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.363976 12248 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:43.364517 12248 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:43.364776  9862 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:43.352566  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:43.379150  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.379511 12125 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:43.380101  9117 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:43.391448  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.391705  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.392545  9853 raft_consensus.cc:2393] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:43.392453  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:43.393450  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.393791  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.394147  9117 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
W20260709 13:04:43.382045  9194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1941) took 2676 ms (client timeout 2462 ms). Trace:
I20260709 13:04:43.393929  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:43.396422  9194 rpcz_store.cc:269] 0709 13:04:40.705200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.459370 (+754170us) service_pool.cc:224] Handling call
0709 13:04:43.381799 (+1922429us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400398354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.381957 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2726561664}
W20260709 13:04:43.400317  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 93899us
W20260709 13:04:43.400851  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.409442  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.410880  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.411871 12071 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:04:43.413113 12142 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:43.413645 12142 raft_consensus.cc:697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:43.414589 12142 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.417532  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.419701  9856 raft_consensus.cc:2410] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:43.424787  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.407855 12150 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:43.428020  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:43.430636  9190 raft_consensus.cc:2410] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:04:43.429633  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:43.432793  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.435673  9120 raft_consensus.cc:2410] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:04:43.436597  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.436650 12291 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:43.444691  9117 raft_consensus.cc:2410] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:04:43.443218 12154 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:43.452477  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 96207.9us
I20260709 13:04:43.453959  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:43.456744  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 133880us
W20260709 13:04:43.457657  9826 leader_election.cc:336] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.032s (SENT)
I20260709 13:04:43.458046  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:43.458249  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.464740  9856 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:43.424955 12266 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:04:43.465122  9120 raft_consensus.cc:2376] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5. Re-sending same reply.
W20260709 13:04:43.471670  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.474766  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.475010 12256 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.482402 12256 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:43.485774  9162 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:43.487218  9162 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:43.488204 12108 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:43.488674 12108 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:43.489060 12108 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:43.489483 12291 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:43.490384  9089 leader_election.cc:336] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.069s (SENT)
I20260709 13:04:43.493697  9854 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.9-6.9]
I20260709 13:04:43.494392  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.494804  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:43.495371  9859 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:43.497728 12256 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.497732 12108 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.498557 12256 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.460012 12071 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.466886  9235 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:43.424710 12248 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:43.464047 12018 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:43.476053  9194 raft_consensus.cc:2410] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:04:43.466741 12062 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:43.476728  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.476514  9121 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:43.479782  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.499787  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:43.506915 12076 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:43.479127  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.506634  9117 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:43.501466  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:43.550912  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.502573  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:43.500272 12108 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:43.503664  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:43.562873  9860 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:43.564396  9190 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:43.563172  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.504477 12248 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:43.563954  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:43.502499 12256 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:43.573374 12069 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:43.504717 12150 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:43.505067 12062 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:43.576136 12062 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:43.571961 12227 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:43.580588  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:43.581233  9190 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:43.583532 12245 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:43.584239  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.584548  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2173) took 4492 ms. Trace:
I20260709 13:04:43.585000  9265 rpcz_store.cc:276] 0709 13:04:39.092500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.114032 (+ 21532us) service_pool.cc:224] Handling call
0709 13:04:39.114320 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:39.708670 (+594350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.584065 (+3875395us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b54001f8954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:43.584361 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1351141248}
I20260709 13:04:43.585700  9194 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:43.587210  9194 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
W20260709 13:04:43.584514  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 123388us
I20260709 13:04:43.500458 12229 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:43.588199  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.588372 12150 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:43.588960  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.589762  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.586916  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.591868  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 1936) took 2888 ms. Trace:
I20260709 13:04:43.591921  9828 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:43.592847  9195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 1925) took 4979 ms (client timeout 2069 ms). Trace:
W20260709 13:04:43.594306  9826 leader_election.cc:336] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.760s (SENT)
I20260709 13:04:43.500129 12071 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.504536 12018 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:43.597128  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 868) took 3261 ms. Trace:
I20260709 13:04:43.722913  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:43.587587  9194 pending_rounds.cc:85] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 6
I20260709 13:04:43.724097  9194 pending_rounds.cc:107] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:04:43.724596  9194 raft_consensus.cc:2889] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:43.699313  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.726187  9854 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 6: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:04:43.603372  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 100935us
I20260709 13:04:43.502944 12262 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:43.702503  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.729526  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:43.729105  9859 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:43.723754  9857 rpcz_store.cc:276] 0709 13:04:40.335380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.335496 (+   116us) service_pool.cc:224] Handling call
0709 13:04:43.247803 (+2912307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:43.273349 (+ 25546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.596925 (+323576us) spinlock_profiling.cc:243] Waited 2.91 s on lock 0x7b540064ffd0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.597071 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6696844672}
W20260709 13:04:43.727895  9195 rpcz_store.cc:269] 0709 13:04:38.613286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:38.613475 (+   189us) service_pool.cc:224] Handling call
0709 13:04:43.565923 (+4952448us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b5400434ed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.592661 (+ 26738us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2166037760}
W20260709 13:04:43.645459  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.728909  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 229319us
I20260709 13:04:43.729550  9197 rpcz_store.cc:276] 0709 13:04:40.703705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.903961 (+2200256us) service_pool.cc:224] Handling call
0709 13:04:42.987225 (+ 83264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.992680 (+  5455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.591607 (+598927us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b5400424fd0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.591773 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18059648}
I20260709 13:04:43.741945  9114 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 13:04:43.742444  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.743130  9854 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:04:43.743155  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:43.744400  9854 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:43.745713  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:43.745481 12062 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.747154  9854 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:43.748405  9854 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:43.750664  9854 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:43.723846 12229 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.753249  9854 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:43.755843 12346 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:04:43.756718  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:43.756860 12258 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:43.757480 12258 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:43.757912 12258 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:43.761058  9118 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 2043) took 2383 ms (client timeout 2050 ms). Trace:
W20260709 13:04:43.761471  9118 rpcz_store.cc:269] 0709 13:04:41.378118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.455652 (+ 77534us) service_pool.cc:224] Handling call
0709 13:04:43.760831 (+2305179us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400265f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:43.760986 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2393338752}
I20260709 13:04:43.763891 12295 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:43.765928 12222 raft_consensus.cc:3055] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:43.766462 12222 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:43.767369 12222 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.768456 12222 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:43.769389  9162 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:43.772625  9856 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
W20260709 13:04:43.774053  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:43.780572  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.782692  9121 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:43.784569  9855 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:43.698858 12064 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:43.785796  9190 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:43.753810 12229 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:43.673400 12069 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.787874  9264 raft_consensus.cc:2410] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:04:43.729985 12262 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:43.787216  9860 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:43.787222  9117 raft_consensus.cc:1217] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-4.3]   Dedup: 2.2->[4.3-4.3]
I20260709 13:04:43.791605  9117 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:04:43.791293  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:43.791966  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 197695us
I20260709 13:04:43.784596 12248 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:43.799276  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:43.800375  9269 raft_consensus.cc:606] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:43.802239 12069 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.803028 12152 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.803922 12257 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.803732 12152 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:43.804792 12069 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.787166 12291 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:43.804504 12257 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:43.812505  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 224028us
I20260709 13:04:43.808015 12115 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:43.814478 12115 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:43.814954 12115 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:43.814491 12078 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:43.816375 12078 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.819271 12078 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:43.820160  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 522) took 3230 ms. Trace:
I20260709 13:04:43.794965 12262 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:43.798851 12062 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:43.798286 12064 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:43.800514 12204 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:04:43.820596  9193 rpcz_store.cc:276] 0709 13:04:40.589328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.589910 (+   582us) service_pool.cc:224] Handling call
0709 13:04:40.604876 (+ 14966us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:40.611068 (+  6192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:40.611105 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:40.646165 (+ 35060us) log.cc:844] Serialized 52 byte log entry
0709 13:04:43.373237 (+2727072us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b18003fe920. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:04:43.406858 (+ 33621us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:43.407226 (+   368us) log.cc:844] Serialized 14 byte log entry
0709 13:04:43.418875 (+ 11649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.425145 (+  6270us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:43.805504 (+380359us) raft_consensus.cc:1685] finished
0709 13:04:43.810656 (+  5152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:43.819953 (+  9297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8259,"prepare.queue_time_us":147,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":65528,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":154574,"spinlock_wait_cycles":3960192,"thread_start_us":22795,"threads_started":1,"wal-append.queue_time_us":2976848}
W20260709 13:04:43.880499  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 376967us
W20260709 13:04:43.891872  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.899801  9116 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:04:43.852473  9118 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:43.924789  9120 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:43.808441 12150 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:43.911458  9828 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:43.860594  9117 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:04:43.931003  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.812144 12248 raft_consensus.cc:697] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:43.932220  9198 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:43.932919  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:43.889020 12257 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:43.940299  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.948999  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.978699  9198 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:43.894991 12282 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:43.973426  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 146489us
I20260709 13:04:43.857268  9121 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:43.848675  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:43.847355  9119 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:43.985950  9193 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:43.892030 12291 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.981493  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:43.990864  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2171) took 4901 ms. Trace:
I20260709 13:04:43.991312  9828 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:43.991835  9198 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:43.831893 12142 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:43.855129 12078 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.138s
I20260709 13:04:43.826740 12256 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.153s
I20260709 13:04:43.994007  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:43.996040 12062 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:44.011612  9266 rpcz_store.cc:276] 0709 13:04:39.089215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.091015 (+  1800us) service_pool.cc:224] Handling call
0709 13:04:39.108891 (+ 17876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:40.127908 (+1019017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:43.990448 (+3862540us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400303ed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:43.990697 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2341977472}
I20260709 13:04:44.000049  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.029346  9854 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:44.029690  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:44.004973 12296 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:43.952811  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:43.948715  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:44.013872  9825 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.030403  9117 raft_consensus.cc:2376] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5. Re-sending same reply.
I20260709 13:04:43.965405  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:43.897495 12115 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.010715 11817 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:43.991856 12160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.046s	sys 0.066s
W20260709 13:04:44.017829  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 169357us
I20260709 13:04:44.019016 12028 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:04:43.992074 12282 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.020051 12064 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.022249 12266 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:44.023125  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:44.024348  9114 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:44.024400  9198 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:04:44.068593  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:44.070006  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:44.076027  9268 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:43.882164 12018 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.025641  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.077932  9859 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:44.026099  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:43.955626 12317 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:04:44.026314  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.027024  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.027617  9121 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:43.997581 12265 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:44.031796 12062 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:44.031906  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.032759  9119 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
W20260709 13:04:44.081421  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 116245us
I20260709 13:04:44.033952 12229 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.035151 12075 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.036298  9857 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:44.036916 12142 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:44.039590  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:44.085323  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.045199 12071 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.049791  9190 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:44.086413  9190 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:43.947736 12280 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:44.077944  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.072247  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.090523  9853 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:44.090665 12069 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:44.091161 12018 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.091305  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:44.092159  9238 connection.cc:256] RPC call timeout handler was delayed by -1.8385970639999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2545s for a response
I20260709 13:04:44.092497  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.093518  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.094233 12245 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:44.094729 12245 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:44.092530  9238 connection.cc:256] RPC call timeout handler was delayed by -1.6081970640000236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2801s for a response
W20260709 13:04:44.096063  9238 connection.cc:256] RPC call timeout handler was delayed by -1.5964970640000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28140000000000004s for a response
W20260709 13:04:44.096385  9238 connection.cc:256] RPC call timeout handler was delayed by -1.4326970640000241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29960000000000003s for a response
W20260709 13:04:44.096818  9238 connection.cc:256] RPC call timeout handler was delayed by -1.2463892320000127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 13:04:44.097098  9238 connection.cc:256] RPC call timeout handler was delayed by -1.1365617449999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 13:04:44.097033 12108 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:44.097631  9238 connection.cc:256] RPC call timeout handler was delayed by -1.1275617450000368s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 13:04:44.097951  9238 connection.cc:256] RPC call timeout handler was delayed by -1.0294617450000487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
W20260709 13:04:44.098484  9238 connection.cc:256] RPC call timeout handler was delayed by -1.0294617450000487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
W20260709 13:04:44.098785  9238 connection.cc:256] RPC call timeout handler was delayed by -1.0213617450000356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
W20260709 13:04:44.099084  9238 connection.cc:256] RPC call timeout handler was delayed by -1.0186892320000425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
I20260709 13:04:44.099706  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.100893  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.071841 12348 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:44.101565 12348 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.101948 12348 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:44.083889  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:44.083022  9118 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:04:44.103981  9271 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:44.083993  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:44.100924  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 160035us
I20260709 13:04:44.094754  9860 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:44.080904  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:44.097914 12108 consensus_queue.cc:1243] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:44.091399  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.113054  9191 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:44.104293  9118 pending_rounds.cc:85] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 3
I20260709 13:04:44.113830  9118 pending_rounds.cc:107] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:04:44.114158  9118 raft_consensus.cc:2889] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:44.116168  9854 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:04:44.105928  9270 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:44.120261  9853 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:04:44.130542  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.132243 12062 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.133368 12245 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:44.133916 12245 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.134327 12245 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.136379  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:44.138211  9270 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:44.139791 12245 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:44.140403 12245 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.140892 12245 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.141636  9268 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:44.142858 12245 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.143369 12245 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.146579  9268 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:44.151108  9860 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:44.151914  9196 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:04:44.154254  9826 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.614s (SENT)
I20260709 13:04:44.100956 12291 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:44.156517  9164 leader_election.cc:336] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Timed out: RequestConsensusVote RPC to 127.7.213.68:43523 timed out after 2.230s (SENT)
I20260709 13:04:44.157022  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:44.157198  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1743) took 4429 ms (client timeout 2433 ms). Trace:
W20260709 13:04:44.157572  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 56759.5us
W20260709 13:04:44.157598  9272 rpcz_store.cc:269] 0709 13:04:39.727545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:39.731297 (+  3752us) service_pool.cc:224] Handling call
0709 13:04:44.156802 (+4425505us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b54002a3d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:44.156999 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2114459392}
I20260709 13:04:44.158464  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:44.158939 12333 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.716s	user 0.000s	sys 0.003s
I20260709 13:04:44.158545 12291 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:44.160179  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 221533us
I20260709 13:04:44.112807 12296 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:44.168757  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 55803.2us
I20260709 13:04:44.167102  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.180174  9121 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:44.117038 12317 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:04:44.154572 12227 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:44.189451  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.190740  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.113230 12071 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:04:44.117264 12280 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.195084  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.196412  9114 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:44.120827 12269 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:04:44.201524  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.113008 12248 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.183609 12108 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:44.215358  9857 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:44.166738 12160 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:04:44.227187  9120 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:44.254879  9271 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:04:44.257236  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 60339.9us
I20260709 13:04:44.204430 12227 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.248881 12291 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:04:44.275998  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 228525us
W20260709 13:04:44.280642  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 160316us
W20260709 13:04:44.285818  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 129348us
I20260709 13:04:44.286922  9161 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:44.197589 12258 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.259665 11817 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:04:44.280170 12082 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:44.227622 12354 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:04:44.269529 12113 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 13:04:44.306741  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 185234us
W20260709 13:04:44.307950  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 207364us
I20260709 13:04:44.310451  9161 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:44.311296  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:44.297079 12291 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:44.331295  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.335108  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56116 (request call id 22) took 1193 ms. Trace:
I20260709 13:04:44.283967 12186 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.193s
I20260709 13:04:44.335552  9269 rpcz_store.cc:276] 0709 13:04:43.134618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.147416 (+ 12798us) service_pool.cc:224] Handling call
0709 13:04:44.327404 (+1179988us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54000f9bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566de853 00007f765b6ac60a 00007f7654c327d4 00007f7654c32762 00007f7654c32691 00007f7654c3260c 00007f7654c318c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:44.327562 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411669376}
I20260709 13:04:44.296535 12262 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.340194  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:44.340912  9861 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 870) took 3876 ms (client timeout 2814 ms). Trace:
I20260709 13:04:44.341178 11801 consensus_queue.cc:1059] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 13:04:44.341393 11517 raft_consensus.cc:3055] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:44.341743  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:44.344622  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 85626.8us
I20260709 13:04:44.342438  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:44.343921  9861 rpcz_store.cc:269] 0709 13:04:40.338747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:40.338867 (+   120us) service_pool.cc:224] Handling call
0709 13:04:44.192943 (+3854076us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400580f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:44.214971 (+ 22028us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376048896}
I20260709 13:04:44.334779 12154 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:44.362124 12366 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:44.361964 12255 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:04:44.362639 12282 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:44.363520 12291 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.364063 12255 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:04:44.362334  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.344786 11517 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:44.367534 11517 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.370464 12361 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:04:44.370810  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:44.371522 12370 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:04:44.371855  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 220652us
I20260709 13:04:44.373136 12280 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:44.373296 12357 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:44.373799 12356 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.375108 12361 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:44.374406 12356 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.385131  9265 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:44.341331 12113 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:44.298341 12142 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.343302 12082 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:44.392454  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47121.2us
I20260709 13:04:44.408138 12356 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:44.332490 12295 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:04:44.410194  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.410889  9119 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:44.375653 12361 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:44.378954 11801 raft_consensus.cc:3055] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:44.414786 12282 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.420497  9161 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:44.420656  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.378799 11517 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:44.378379 12262 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:44.378626 12366 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:44.374923 12365 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:44.401470 12113 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.378940 12364 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:44.379245 12369 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:44.423425  9269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:04:44.404207 12154 raft_consensus.cc:697] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:44.436370  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 128930us
I20260709 13:04:44.452265  9269 raft_consensus.cc:606] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.452602  9269 raft_consensus.cc:614] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:44.455034 12361 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:44.464597 12352 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:04:44.465930  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:44.473834  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 53403.1us
W20260709 13:04:44.475610  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54446.2us
I20260709 13:04:44.418663 11801 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:44.477569  9162 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:44.481094  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.450098 12113 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.452003 12365 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.485230  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.450215 12364 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:44.488258  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:44.472800 12358 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:44.502573  9115 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:44.486567 12360 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.504745 12360 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.507300 12360 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.477303 12262 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:04:44.424307 12108 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:04:44.476807 12258 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:44.493891 12376 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:44.495092  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 83084.4us
I20260709 13:04:44.424635 12282 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.477082 12291 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:04:44.503552 12167 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.547948  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.488232 12064 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:44.555176  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 77712.7us
W20260709 13:04:44.575102  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 106553us
I20260709 13:04:44.487138 12365 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:44.579455  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.563536 12369 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:44.580117 12355 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:04:44.581816  9825 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.584045 12154 consensus_queue.cc:237] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.584261 11801 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.580164  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:44.585454 11801 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:44.476693 12082 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.584071 12369 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:44.584098  9825 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.546681 12245 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:44.548519 12356 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:44.557397 12376 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:04:44.606788  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 118943us
I20260709 13:04:44.555127 12382 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:04:44.547803 12348 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.597558 12360 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:44.601012 12265 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.612850  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.602190 12369 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:44.621376 12361 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:44.621863  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.599900 12383 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:44.622886  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:44.601239 12376 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:44.624668  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.625135 12289 raft_consensus.cc:2764] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:04:44.626135  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.627254  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.629279  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.622531 12142 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:44.637661 12245 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:44.638150 12245 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.638535 12245 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.598595 12167 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.639843 12348 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:44.640105 12371 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:44.645501 12076 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.646034 12076 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.630160 12064 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:44.605842 12366 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.612146 12364 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.653123 12364 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:44.634739 12383 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:44.657881  9861 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:44.659025 12282 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.659929  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2108) took 4004 ms. Trace:
I20260709 13:04:44.660293  9122 rpcz_store.cc:276] 0709 13:04:40.655262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.824381 (+2169119us) service_pool.cc:224] Handling call
0709 13:04:42.899620 (+ 75239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:42.955439 (+ 55819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:42.955507 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:42.964238 (+  8731us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:42.968248 (+  4010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:42.968328 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.659717 (+1691389us) spinlock_profiling.cc:243] Waited 57 ms on lock 0x7b5400254250. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:44.659854 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131005312}
I20260709 13:04:44.662336 12265 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:44.665431  9087 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.543s (SENT)
I20260709 13:04:44.665973  9087 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:44.666839 12296 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.667101  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.667338 12296 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:44.667907  9123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2119) took 1262 ms (client timeout 1631 ms). Trace:
W20260709 13:04:44.668253  9123 rpcz_store.cc:269] 0709 13:04:43.406029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.666813 (+1260784us) service_pool.cc:224] Handling call
0709 13:04:44.667850 (+  1037us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:44.669392 12296 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:44.669526 12113 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.674378  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.674937  9857 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:44.672739  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.674436 12360 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:04:44.678311 12389 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:44.679278 11517 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:44.679795 11517 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.681660 11517 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.683559  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:44.684226  9861 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:44.684721  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:44.685376 12076 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.685751  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:44.686331  9198 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:44.689035  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:44.687494  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 49311.2us
I20260709 13:04:44.692332  9825 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:44.693902 12383 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.694564  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:44.695182  9197 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:44.695681  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 913) took 3559 ms. Trace:
I20260709 13:04:44.696067  9858 rpcz_store.cc:276] 0709 13:04:41.136291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:41.266557 (+130266us) service_pool.cc:224] Handling call
0709 13:04:44.201032 (+2934475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:44.201357 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:44.201381 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:44.201433 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:44.201481 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:44.201538 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:44.695423 (+493885us) spinlock_profiling.cc:243] Waited 2.93 s on lock 0x7b5400580f50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:44.695529 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6745526528}
I20260709 13:04:44.693930  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.698872  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.692247 12265 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:44.699992 12369 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:44.700434 12369 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:44.700781 12369 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.704267 12150 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:44.705966  9193 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:44.706732 12113 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.706004  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:44.717942  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 52538.6us
W20260709 13:04:44.718534  9087 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.616s (SENT)
I20260709 13:04:44.722293 12277 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:44.725143  9190 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.729557  9190 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:44.730178 12353 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.529s	user 0.003s	sys 0.004s
W20260709 13:04:44.731840  9115 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 525) took 1489 ms (client timeout 1692 ms). Trace:
W20260709 13:04:44.732241  9115 rpcz_store.cc:269] 0709 13:04:43.242011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.243658 (+  1647us) service_pool.cc:224] Handling call
0709 13:04:44.731611 (+1487953us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400120a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:44.731782 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2887942912}
I20260709 13:04:44.733470  9118 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:04:44.734737 12371 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:44.746376  9190 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:04:44.749534  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:44.754011  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:44.755278  9825 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.719672 12265 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:44.759475  9164 leader_election.cc:336] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Timed out: RequestConsensusVote RPC to 127.7.213.68:43523 timed out after 1.586s (SENT)
W20260709 13:04:44.760790  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 68276us
I20260709 13:04:44.760492 12360 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:44.692550 11801 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:04:44.715348 12376 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.764906  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.756783 12277 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:04:44.772300  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 53752.6us
I20260709 13:04:44.760150 12265 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.764560 12371 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:44.766260 12360 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.767980 12277 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.783362 12229 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.784142 12257 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:44.784879 12295 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:44.785102 12257 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:44.786217 12257 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.787482 12257 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:44.775782 12028 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:44.790460 12062 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:44.791790  9825 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:44.792227  9087 leader_election.cc:336] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.691s (SENT)
I20260709 13:04:44.792726  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:44.793696  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.794394 12076 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:44.796751 12152 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:44.798981 12204 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:44.799388 12204 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:44.778576 12360 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:44.800335 12204 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.801347 12204 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:44.802484  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.774235 12265 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.780184 12277 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.810405  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:44.810984  9122 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.811424  9117 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:44.768234 12142 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:04:44.815083  9197 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 13:04:44.815843  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.816010  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:44.765609 12369 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.817888  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:44.818596  9190 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:44.817952 12388 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:04:44.820399  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.807993 12062 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:44.825310 12245 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:44.827564  9264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2179) took 3310 ms (client timeout 1721 ms). Trace:
W20260709 13:04:44.835325  9264 rpcz_store.cc:269] 0709 13:04:41.516706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:42.729048 (+1212342us) service_pool.cc:224] Handling call
0709 13:04:44.827382 (+2098334us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540032e454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:44.827503 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2427617664}
I20260709 13:04:44.833033  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:44.834522  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:44.836401  9119 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.837276 12355 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:44.837972  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:44.838908  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.839699  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.840346  9115 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:44.841544 12369 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:44.843129 12018 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:44.843525 12018 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:44.844360 12018 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.812801 12152 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:44.845368 12018 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.847291 12028 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:44.855464  9235 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:44.856873  9235 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:44.868592 11801 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:44.828871 12365 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.833563  9825 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:44.876353  9119 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
W20260709 13:04:44.870549  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:44.878152  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:44.835186  9122 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:44.880548 12282 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:44.882989  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 50033.8us
I20260709 13:04:44.883735  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.884522 12121 consensus_queue.cc:1059] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:44.885546 12121 raft_consensus.cc:3055] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:44.885865 12121 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:44.886488 12282 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:44.883275  9194 raft_consensus.cc:2368] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:04:44.888195  9190 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:04:44.890887  9267 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:44.891356 12222 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:44.891785 12222 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:44.892685 12222 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.893286 12115 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.887655  9235 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.895184 12222 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.897900  9235 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:44.877271 12395 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:04:44.900341  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 1964) took 1319 ms. Trace:
I20260709 13:04:44.900679  9191 rpcz_store.cc:276] 0709 13:04:43.580546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:43.587936 (+  7390us) service_pool.cc:224] Handling call
0709 13:04:44.900166 (+1312230us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b540010c554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:44.900283 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":881197312}
I20260709 13:04:44.901631 12167 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.902549  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:44.903105  9269 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.895787  9828 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:44.903576 12160 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:44.904747 12160 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:44.905088  9828 leader_election.cc:400] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:04:44.905408  9828 leader_election.cc:403] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:04:44.905627 12160 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.906536 12062 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:04:44.906836 12160 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:44.906080 12069 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:44.908071 12069 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:44.908933 12069 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:44.912153  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 94382us
I20260709 13:04:44.873317 11801 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.921311  9269 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:44.920408 12291 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:44.924629 12291 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:44.925110 12291 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:44.920554 12374 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:44.927891  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:44.925355  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.928857  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:44.937412 12365 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.929628 12282 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:04:44.922326 12348 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:44.941745 12348 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:44.930117 11801 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.942215 12121 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.944099  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:44.944804  9264 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:44.949043 12245 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.942282 12348 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.952955 12265 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.917748 12356 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:44.967839  9856 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:04:44.968793  9856 pending_rounds.cc:85] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 6
I20260709 13:04:44.969146  9856 pending_rounds.cc:107] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:04:44.969419  9856 raft_consensus.cc:2889] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:44.959705  9087 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.859s (SENT) [suppressed 54 similar messages]
I20260709 13:04:44.971679  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:44.972023  9117 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:04:44.973577  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:44.974345 12204 consensus_queue.cc:1059] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:44.974767  9164 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:44.975502 12204 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.955721 12265 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.976511 12258 raft_consensus.cc:3055] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:44.976573 12376 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:44.976975 12258 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:44.977257 12376 raft_consensus.cc:2764] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:04:44.977903 12258 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.979055 12076 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:44.979434 12076 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:44.979482 12258 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:44.980357 12076 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:44.980592 12062 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:04:44.980618 12365 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:44.968724 12356 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.001955 12356 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.986083  9121 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:44.970369  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:44.960292 12277 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:44.992210 12076 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:44.988951 12121 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:44.993060  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 67830.2us
I20260709 13:04:44.995844 12064 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:44.977941 12376 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:04:45.016376 12142 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:45.018247 12227 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:45.029341  9264 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:04:45.032179  9861 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:04:44.929352 12360 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:45.036298  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 113706us
W20260709 13:04:45.037170  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.040391  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:45.044800  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:45.051057  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:45.051162  9860 raft_consensus.cc:2376] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7. Re-sending same reply.
I20260709 13:04:45.057978  9191 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:45.059284  9862 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:45.072227 12389 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:45.072746 12389 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:45.073544 12389 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.074564 12389 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:45.049566 12374 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:45.076198 12374 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:45.077348  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:45.077099 12374 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.077919  9121 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:45.049868  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:45.080389  9196 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:45.080678  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:45.081544  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:45.081846 12062 consensus_queue.cc:1059] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:45.082649  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:45.082830 12062 raft_consensus.cc:3055] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:45.083223  9853 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:45.083285 12062 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:45.084312 12062 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.084635  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:45.085188 12062 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:45.085736  9858 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:45.086692  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:45.072762  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:45.092861 12277 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:45.093371 12355 raft_consensus.cc:2764] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:04:45.094043 12355 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:45.085191  9860 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:45.096436  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:45.098218  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:45.082147  9116 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 13:04:45.099256  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:45.105751  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:45.106081  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72932.2us
I20260709 13:04:45.106207  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:45.106905  9119 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:45.095191 12152 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:45.110285  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:45.114696  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:45.115289  9271 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:45.116392  9861 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:45.117580  9189 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:04:45.117590  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:45.114935  9862 raft_consensus.cc:2376] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7. Re-sending same reply.
I20260709 13:04:45.117924  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:45.127665  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:45.128058  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.129387  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:45.127687  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:45.130211  9858 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:45.130375  9265 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:45.132102  9860 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:04:45.129973  9854 raft_consensus.cc:2376] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8. Re-sending same reply.
I20260709 13:04:45.133913  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:45.145298  9861 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 11 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:04:45.138046  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:45.146128  9271 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:04:45.147199  9855 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:45.136643  9121 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:04:45.110059 12371 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:45.152499  9861 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:45.058064 12245 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:04:45.124567 12383 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.159884  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 103093us
I20260709 13:04:45.162788  9859 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:04:45.167296  9116 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:45.133877 12277 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:04:45.168605  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 75743.9us
I20260709 13:04:45.169646  9164 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:45.157567 12204 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:45.180687  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:45.181799  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.183589  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:45.188997 12382 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.189546 12382 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:45.167696 12366 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:45.191953 12108 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.183324 12277 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:45.192530 12108 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.193634 12277 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.194732 12108 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:45.166013 12371 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.198184  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:45.198410 12371 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:45.200248 12113 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:45.200971 12113 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:45.202075 12113 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.170608 12383 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:45.205612 12383 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:45.182865 11517 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.214669  9087 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.115s (SENT)
I20260709 13:04:45.215173  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:45.191538 12382 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:45.236518  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.240294  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:45.192204 12366 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:04:45.245147  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 75556.1us
I20260709 13:04:45.245649 12361 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:45.246713  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:45.257490  9189 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:45.241156  9116 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:45.258922  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:45.228191 12064 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:45.262718  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:45.269814  9120 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:04:45.270357  9826 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.627s (SENT)
I20260709 13:04:45.273372  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:45.274839 12372 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:45.277498  9120 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:04:45.281466  9120 pending_rounds.cc:85] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 3
I20260709 13:04:45.282595  9120 pending_rounds.cc:107] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 13:04:45.283087  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.284281  9120 raft_consensus.cc:2889] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:45.224598 11517 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:45.297387  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 82753.3us
I20260709 13:04:45.294873  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:45.300349  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:45.260663 12360 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:45.211253 12291 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:45.284312  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 110803us
W20260709 13:04:45.310930  9826 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.712s (SENT)
I20260709 13:04:45.311671  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:45.231953 12018 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:04:45.261579 12386 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:45.355233  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44357.2us
I20260709 13:04:45.293162 12064 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:45.364663  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 123459us
I20260709 13:04:45.367141  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:45.425035  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 58005.6us
I20260709 13:04:45.357240 12018 consensus_queue.cc:1243] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:04:45.352283 12258 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:45.427541  9162 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:45.305476 12382 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:04:45.370790 12064 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:45.356645 12062 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:45.426281 12028 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:04:45.441710  9162 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:45.317464 12348 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.447718  9853 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
W20260709 13:04:45.448758  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.443673  9120 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:45.428470 12282 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:45.459163 12172 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:45.459394 12283 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:45.459779 12172 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.460203 12172 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:45.460222  9116 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:45.430359 12365 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:45.457907 12258 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:45.496668 12258 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:45.497159 12291 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:45.497798 12258 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.427307 11801 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:45.500978 12408 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:04:45.502190 11801 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:45.502385 12399 log.cc:927] Time spent T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 13:04:45.455555 12372 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.503163 12372 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.505707 12372 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:45.472231 12378 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:04:45.475541 12365 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:45.508667 12397 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.584s	user 0.005s	sys 0.000s
I20260709 13:04:45.508581 12365 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:45.475541 11517 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:45.484851  9196 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:45.514976  9189 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
W20260709 13:04:45.447683 12160 consensus_meta.cc:287] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.000s
W20260709 13:04:45.473557 12402 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
W20260709 13:04:45.473033 12405 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 13:04:45.486635 12403 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:04:45.513154 12404 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:04:45.536038  9857 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:45.474177 12229 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:45.545296  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:45.552577  9826 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.692s (SENT)
I20260709 13:04:45.555217  9196 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:45.445801 12366 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.603880  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:45.604496  9121 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:04:45.605321  9115 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:45.588485 12229 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:45.610929 12389 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.000s
I20260709 13:04:45.630999  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:45.633781  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.631299  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:45.629204 12356 consensus_meta.cc:287] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.002s
W20260709 13:04:45.644752  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 338354us
W20260709 13:04:45.632514  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 190882us
W20260709 13:04:45.649497  9087 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.208s (SENT)
I20260709 13:04:45.652132  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.652863  9271 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:45.643471 12075 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:45.648998  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:45.654675  9826 leader_election.cc:336] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.060s (SENT)
I20260709 13:04:45.655192  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:45.621554 12348 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:45.661128  9087 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:45.642904 12142 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:04:45.664332  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.672271  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:45.673480  9269 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:04:45.673938  9269 pending_rounds.cc:85] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 2
I20260709 13:04:45.635082 12386 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:04:45.679157  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.674216  9269 pending_rounds.cc:107] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:04:45.680160  9269 raft_consensus.cc:2889] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:45.670008  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.683312  9265 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:04:45.683924  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 330992us
I20260709 13:04:45.675400  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:45.653645 12075 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.670583  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.676388  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.685550  9271 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:45.688135  9263 raft_consensus.cc:1217] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Deduplicated request from leader. Original: 3.4->[5.5-9.8]   Dedup: 5.5->[5.6-9.8]
I20260709 13:04:45.688763  9263 pending_rounds.cc:85] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 5
I20260709 13:04:45.689067  9263 pending_rounds.cc:107] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:04:45.619683 12064 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.662930 12018 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:45.668006 12360 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:45.644191 12416 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:04:45.679325 12369 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:45.689287  9263 raft_consensus.cc:2889] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:45.691473  9264 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:45.654320 12356 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:45.671809 12142 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:45.706835  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.717617  9267 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:45.686211 12075 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:45.648552 12172 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:45.667909 12389 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
W20260709 13:04:45.718309  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 68842us
I20260709 13:04:45.720783  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:45.722131 12265 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.692624 12277 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:45.723922  9270 raft_consensus.cc:2368] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5.
I20260709 13:04:45.724937  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:45.725490  9114 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:45.694888 12018 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.726774  9266 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:45.731299 12265 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.731683  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.732825  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:45.733592 12265 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:45.734287  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.735221  9268 raft_consensus.cc:2410] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:45.728875 12018 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:45.736901 12282 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:45.737365 12282 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:45.741228 12108 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:45.741598 12277 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:45.742694 12108 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:45.742533 12277 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.743552 12064 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:45.622578 12229 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.752429  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:45.753181  9860 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:45.753438  9268 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:04:45.753824  9268 pending_rounds.cc:85] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 8
I20260709 13:04:45.754065  9268 pending_rounds.cc:107] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:04:45.754346  9268 raft_consensus.cc:2889] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:45.755352  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.755806  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:45.758400  9264 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:04:45.771425  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.780458  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:45.781267  9268 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:45.788666  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:45.752496 12069 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:45.761304 12283 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:45.793911  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 65387.4us
I20260709 13:04:45.771162 12386 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.803727  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.810462  9264 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:45.820207  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:45.825349  9826 leader_election.cc:336] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.231s (SENT)
I20260709 13:04:45.825857  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:45.835731  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.805815 12269 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:04:45.840085  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:45.843952  9087 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.709s (SENT)
I20260709 13:04:45.844663  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:45.846092 12277 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:45.848505  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:45.866040  9826 leader_election.cc:336] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.947s (SENT)
I20260709 13:04:45.871515  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:45.872457  9115 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:45.876859  9087 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.433s (SENT)
I20260709 13:04:45.814085 12386 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:45.809603 12069 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.811501 12283 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.881107  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.882143  9264 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:45.840516 12369 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:45.891718  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.840454  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:45.884809 12277 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.833492 12142 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:45.904369 12428 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:04:45.904943 12430 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:04:45.906635 12018 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:45.908257 12283 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:45.909756 12069 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:45.910604 12424 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:04:45.910998 12172 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:45.911303 12386 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:45.898123 12356 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:45.937915  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:45.945225  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:45.994745  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.952767  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:45.995944  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:45.996428  9196 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:45.953401  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:45.957340 12018 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:45.958285 12075 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:45.962874  9115 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:45.965947  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.995494  9266 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:45.969584 12277 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:45.974056 12372 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:45.974947  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.004173  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:46.004377  9271 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:45.980705  9114 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:45.987488  9860 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:04:45.971654 12431 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.158s	user 0.002s	sys 0.000s
I20260709 13:04:45.997293  9858 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:46.008193  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:45.985205 12361 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:45.996752 12386 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.009642  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.009681 12386 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:46.010859  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:46.012032  9164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.013s	sys 0.009s
W20260709 13:04:46.012460  9164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.013s	sys 0.009s
I20260709 13:04:46.012108 12372 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:46.012992  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 718183us
I20260709 13:04:46.013711 12277 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:46.014485 12372 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:46.015890 12283 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:46.016523  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.016891  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.017287  9272 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:45.999038  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:46.017786  9271 raft_consensus.cc:2410] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:04:46.017928  9120 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:46.018056  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:45.911597 12282 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.018761  9271 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:46.019924  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.020026 12282 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:46.006404  9235 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:46.021588 12386 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:46.021829 12282 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:46.022377 12282 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:46.022727  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:46.023541  9196 raft_consensus.cc:2410] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:46.023767  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:46.024519  9858 raft_consensus.cc:2410] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:46.021612  9235 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:46.020787 12277 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:46.009476  9069 placement_policy_util.cc:248] tablet 8a4bd074bf594b89b28b47d7482e8ac1: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 13:04:46.028839  9271 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:46.029841  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.030424  9266 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:46.012295  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.030260  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:46.018769  9272 raft_consensus.cc:2393] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:46.028375 12277 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.032351 12277 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:46.034327 12075 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:46.036962 12172 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.039292  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.040117  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.039175  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.040390  9271 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:46.040925  9267 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:46.042047  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.043334  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.044564  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:46.047005  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.050613  9264 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:46.045536  9267 raft_consensus.cc:2410] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:04:46.054661  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:46.037472 12172 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:46.061110 12062 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.061231  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:46.061925  9119 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:46.061729 12062 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.059327 12265 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:46.064792 12062 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:46.070040  9119 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:04:46.059397 12450 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:46.076299  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:46.058795 12282 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:46.053486 12069 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:04:46.081866 12069 consensus_queue.cc:1243] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 6.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:46.082365 12069 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:46.082677 12069 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:46.085072  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.086535  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.086905  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.087872  9272 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:46.091203  9266 raft_consensus.cc:2393] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:46.093958  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.095027  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.057667 12355 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:46.100033  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:46.150174  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.100683  9271 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:46.173022  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.175230  9272 raft_consensus.cc:2410] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:04:46.175266  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.176579  9271 raft_consensus.cc:2376] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9. Re-sending same reply.
I20260709 13:04:46.179651  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.180413  9264 raft_consensus.cc:2376] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9. Re-sending same reply.
I20260709 13:04:46.178267  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.186170  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:46.188066  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 129775us
I20260709 13:04:46.191957  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.192775  9264 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:46.083521 12069 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.190027 12376 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:46.225133  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 134615us
I20260709 13:04:46.245044  9266 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:46.227314  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:46.228233 12258 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:46.181969 12355 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:46.244129 12062 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:46.265900 12062 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.268702 12355 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:46.273420 12453 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:04:46.274849  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 47612.5us
W20260709 13:04:46.275408  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 199180us
I20260709 13:04:46.252434 12258 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.286983  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:46.287245 12113 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:46.298683 12062 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:46.300007 12227 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:46.291791  9272 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:46.301630  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 529) took 1313 ms. Trace:
I20260709 13:04:46.302014  9123 rpcz_store.cc:276] 0709 13:04:44.988583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:44.988700 (+   117us) service_pool.cc:224] Handling call
0709 13:04:45.013994 (+ 25294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:45.014376 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:45.014407 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:45.014842 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:04:45.122875 (+108033us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:45.148594 (+ 25719us) log.cc:844] Serialized 14 byte log entry
0709 13:04:45.148775 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:45.148854 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:46.298655 (+1149801us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b18003c4f00. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:46.301289 (+  2634us) raft_consensus.cc:1685] finished
0709 13:04:46.301470 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:46.301579 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":49428,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":329,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":7361,"spinlock_wait_cycles":5862656,"thread_start_us":107681,"threads_started":1,"wal-append.queue_time_us":156070}
I20260709 13:04:46.304133  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:46.305115  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.266258 12365 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.305737 12018 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:46.306305 12113 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:46.314193 12113 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:46.315697  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.288156 12258 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:46.320505  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.321105  9263 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:46.321985  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.322986  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.307404  9164 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:46.274246 12069 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:46.252611 12376 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:46.310225  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.369095  9264 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:46.370692 12456 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:04:46.311528  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.317472 12365 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.308588  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.336781 12277 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.343338  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:46.423622  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.388119 12411 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.811s	user 0.004s	sys 0.000s
I20260709 13:04:46.389175  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:46.501519  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 194179us
I20260709 13:04:46.414342  9268 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
W20260709 13:04:46.505434  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.504364  9269 raft_consensus.cc:2368] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:04:46.506093  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:46.507071  9264 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:04:46.541558  9826 leader_election.cc:336] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.645s (SENT)
W20260709 13:04:46.531558  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 187107us
I20260709 13:04:46.561096  9189 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 13:04:46.562096  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:46.562920  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.563422  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:46.384296  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:46.455694  9087 leader_election.cc:336] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.513s (SENT)
I20260709 13:04:46.555931  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:46.564257  9196 raft_consensus.cc:2376] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8. Re-sending same reply.
I20260709 13:04:46.564502  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:46.552588  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:46.390704 12419 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.706s	user 0.000s	sys 0.002s
I20260709 13:04:46.565632  9272 raft_consensus.cc:2368] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
W20260709 13:04:46.565655  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 110012us
I20260709 13:04:46.566259  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:46.568161  9121 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:46.559520  9860 raft_consensus.cc:2376] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8. Re-sending same reply.
W20260709 13:04:46.568827  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.568222  9263 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:46.368185 12018 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:46.574885  9121 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:46.575248  9859 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:46.577534  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:46.578047  9198 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:46.572222  9162 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:46.580586  9858 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:46.583624  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:46.580031  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:46.584939  9118 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:04:46.585302  9118 pending_rounds.cc:85] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 3
I20260709 13:04:46.585515  9118 pending_rounds.cc:107] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:04:46.585696  9121 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:46.585731  9118 raft_consensus.cc:2889] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:46.587239  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.422785 11517 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.587623  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:46.586666  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:46.589597  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.589706  9198 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:46.590204  9268 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 3.
I20260709 13:04:46.588339  9860 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:46.591195  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:46.591830  9192 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:46.388311 12365 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:46.595124  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:46.575489 12369 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:46.595842  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.381278 12258 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.597415  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.608150  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:46.608964  9263 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:46.598491  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:46.600118  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:46.604007  9264 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:46.608081  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:46.610594  9197 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:46.575842 12076 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:46.614192  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:46.615455  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.618216  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:46.609066  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:46.622031  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 47429.2us
W20260709 13:04:46.622480  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 50366.7us
I20260709 13:04:46.623488  9162 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:46.626099  9189 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:46.628038  9189 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:46.632874  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.633687  9272 raft_consensus.cc:2376] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5. Re-sending same reply.
I20260709 13:04:46.634657  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.636047  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:46.464118 12442 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:46.639183  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.640756  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:46.599349 12062 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:04:46.647205  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:46.641824  9859 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:46.649734  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:46.643553  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:46.650745  9855 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:46.651149  9853 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:46.646114 12420 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:46.654004  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:46.655169  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.662719  9192 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:46.656217  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.611930 12366 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:46.664474  9271 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:46.654968  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.692996  9263 raft_consensus.cc:2368] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 6. Current term is 7.
I20260709 13:04:46.654541  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.659590  9196 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:46.657734  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.658891  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.748898  9197 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:46.646266 12142 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:46.651702 12371 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.022s	sys 0.074s
I20260709 13:04:46.751937  9853 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:04:46.753945  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.648217 12172 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:46.639583 12265 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.113s
I20260709 13:04:46.759907  9859 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:04:46.762392  9271 raft_consensus.cc:2410] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:46.609584 11517 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.781720  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:46.795802  9268 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:46.858016  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:46.760667 12371 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.022s	sys 0.074s
I20260709 13:04:46.642318 12369 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.667268 12121 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.203s
I20260709 13:04:46.855423 12441 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.857203 12269 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:46.655479 12167 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.222s
I20260709 13:04:46.892696  9269 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:46.900208  9269 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:46.893862  9268 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:46.890501  9264 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:04:46.908618 12258 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:46.916244 12365 raft_consensus.cc:2955] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:46.659655 12442 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:46.928575  9114 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:46.930581 12442 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:46.932145 12160 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:46.932744 12160 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.934937 12160 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:46.901342  9269 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.936008  9269 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:46.884119 11805 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.944172 11805 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:46.944954  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:46.946525 11805 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:46.948501 11517 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:46.955024  9855 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:04:46.751735 12366 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:46.957327  9264 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:46.959354  9192 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:46.963313 12018 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:46.964069 12442 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:46.964267 12071 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.966696 12369 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:46.899547 12441 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:46.967806 12361 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:47.012372 12442 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:47.017071  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 446398us
I20260709 13:04:47.030181 12369 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:47.030894 12369 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:47.031384  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 424530us
I20260709 13:04:47.033416 12369 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.034653 12369 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:47.035719 12369 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.044559  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.045460  9271 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:47.017268 12361 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:47.014343 12071 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:47.089995 12069 consensus_meta.cc:287] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.713s	user 0.007s	sys 0.003s
I20260709 13:04:47.091099 12071 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.046252 12152 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:04:47.096273  9194 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 4. Current term is 5. Ops: []
W20260709 13:04:47.105154  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 465656us
I20260709 13:04:47.106132 12071 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:47.106418 12441 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.103193 12366 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:47.046258 12355 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:04:47.109776  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.111404  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.114312  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.022265  9269 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:04:47.115410  9269 pending_rounds.cc:85] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 8
I20260709 13:04:47.115795  9269 pending_rounds.cc:107] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:04:47.116155  9269 raft_consensus.cc:2889] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:47.117974  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:47.112864  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 457519us
I20260709 13:04:47.120823  9087 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:47.110920  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 487511us
I20260709 13:04:47.123791  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.119506 12441 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.118148 12366 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.124233 12283 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:47.134641  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.125699 12142 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.133034 12121 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:04:47.098716 12442 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.125988 12386 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:04:47.148830 12269 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:47.141588 12366 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.154048 12442 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:47.157433 12361 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.158840  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.122850  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:47.164352  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.180138  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 57340.1us
I20260709 13:04:47.182829  9855 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:04:47.166726 12426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.324s	user 0.011s	sys 0.026s
I20260709 13:04:47.189065 12366 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.199167  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.199970  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:47.200304  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:47.202214  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.182586 12436 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.182367 12142 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:47.184957 12426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.345s	user 0.011s	sys 0.026s
I20260709 13:04:47.185951 12474 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:47.214057  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 102500us
I20260709 13:04:47.216084  9087 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.218952  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:47.221342  9265 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:47.205421 12283 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.205720 12152 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:47.223542 12152 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.223933 12152 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:47.227020 12361 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.214018 12282 consensus_queue.cc:1059] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:04:47.229316 12436 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:04:47.230614 12436 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:47.231117 12436 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:47.231213 12361 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:47.233464 12152 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.230521  9087 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:47.236289 12283 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:47.236745 12283 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:47.237267 12283 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:47.238600  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.239708  9087 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:47.240525 12441 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:47.245301  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.246842  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.248653  9087 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.250221  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.218556  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2235) took 1426 ms. Trace:
I20260709 13:04:47.251245  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:04:47.214938 12467 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:04:47.203653 12355 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:47.251272  9270 rpcz_store.cc:276] 0709 13:04:45.792268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:45.792719 (+   451us) service_pool.cc:224] Handling call
0709 13:04:45.811689 (+ 18970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:45.811967 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:04:47.085906 (+1273939us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b1800429340. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f765628c4bc 00007f76535da48b 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f4455 00007f765a0dc3c7 00007f76566ee9a7 00007f76566fe0bf
0709 13:04:47.086086 (+   180us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.217955 (+131869us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.218038 (+    83us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:47.218108 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.218295 (+   187us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.218475 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10861,"prepare.queue_time_us":66310,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":17998,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"spinlock_wait_cycles":3573632,"thread_start_us":196189,"threads_started":3,"wal-append.queue_time_us":141734}
I20260709 13:04:47.211123 12389 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:47.235929 12152 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:47.273104  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54242.3us
I20260709 13:04:47.266451  9826 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.277143  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.277755  9826 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.278086  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.279264  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:04:47.277143  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:47.277766  9264 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:47.278705  9272 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:47.282513  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.255162 12470 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:47.253526 12371 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.259585 12142 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:47.253367 12366 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:47.254179 12472 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:47.328365  9087 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.334960  9198 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:47.305054  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.358615  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.363273  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.368242  9268 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:04:47.304171  9192 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:04:47.359301  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.370918  9264 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:47.279404  9267 raft_consensus.cc:2955] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:47.377383  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.378544  9272 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:47.386205  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.391127  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:47.391887  9198 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:47.266866 12468 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:47.257167 12265 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:04:47.272763 12494 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:47.280612 12476 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:47.303611 12069 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:47.294695 12348 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:47.411266  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.411147  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.411844  9196 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:04:47.412360  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 129940us
W20260709 13:04:47.403614 12480 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:04:47.256846 12441 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:04:47.427973  9192 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:04:47.385691 12258 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.268797 12291 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.136s
I20260709 13:04:47.303283 12470 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:47.249993 12474 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.368674 12472 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.436146  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.426790  9190 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:47.361106 12366 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:47.280838 12172 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.131s
W20260709 13:04:47.368690 12467 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:04:47.404850 12371 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:47.424193  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 95842us
I20260709 13:04:47.483918  9087 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.455303  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:47.432622 12355 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:47.434371 12494 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:47.497823  9190 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:47.431537 12069 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:47.479287 12486 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
W20260709 13:04:47.484079 12487 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:04:47.464053 12366 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:47.440016 12476 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:47.272778 12436 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.446550 12470 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:47.544157  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.446659 12472 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:47.496558 12348 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.548537  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:47.548574  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 292997us
I20260709 13:04:47.500573 12504 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:47.456277 12389 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:04:47.551867  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 692) took 1234 ms. Trace:
I20260709 13:04:47.552824  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.533277 12142 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.453111 12258 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.535646 12501 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:47.558748  9263 rpcz_store.cc:276] 0709 13:04:46.316947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.876936 (+559989us) service_pool.cc:224] Handling call
0709 13:04:46.890381 (+ 13445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.890635 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:47.168970 (+278335us) log.cc:844] Serialized 14 byte log entry
0709 13:04:47.551376 (+382406us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:47.551443 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:47.551506 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.551627 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.551800 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25089,"prepare.queue_time_us":324121,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":4723,"raft.queue_time_us":37031,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":65504,"thread_start_us":619874,"threads_started":3,"wal-append.queue_time_us":330252}
I20260709 13:04:47.559859  9268 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:47.548086 12018 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.514654 12069 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:47.576948  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 93145.1us
I20260709 13:04:47.577625  9162 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:47.591312  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.585922 12283 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.514817 12355 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.594132  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 901) took 1014 ms. Trace:
I20260709 13:04:47.592368  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:47.596813  9861 rpcz_store.cc:276] 0709 13:04:46.579388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.581386 (+  1998us) service_pool.cc:224] Handling call
0709 13:04:46.581663 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.014954 (+433291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.014998 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.015592 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:04:47.207126 (+191534us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:47.207178 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.207300 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:47.593944 (+386644us) raft_consensus.cc:1685] finished
0709 13:04:47.593962 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:47.594081 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17159,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":606687,"threads_started":2,"wal-append.queue_time_us":235544}
I20260709 13:04:47.574805 12436 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.551098 12476 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.551613 12502 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:04:47.607573 12476 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:47.608173 12476 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.608623 12476 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.602703 12227 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:04:47.610541 12476 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:47.610769 12365 consensus_meta.cc:287] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.000s
I20260709 13:04:47.611024 12476 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.603360 12384 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:47.612223 12384 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.612568 12384 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:47.614033  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.616318 12474 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:47.618647 12283 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.604851 12494 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.611505 12476 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.623226 12476 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.592438  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:47.553287 12348 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.624920 12384 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:47.655488 12384 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.569250 12500 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:04:47.631280  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.657300  9266 raft_consensus.cc:2368] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:04:47.636118  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.596000 11805 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:47.637704 12361 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:47.637899  9196 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:47.638504  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.575568 12468 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:47.642987  9198 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
W20260709 13:04:47.644462  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 52092.7us
I20260709 13:04:47.645932  9193 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: []
W20260709 13:04:47.647608  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 97901.1us
I20260709 13:04:47.650815  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.648605 12474 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:47.652587 12355 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.652941 12348 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:47.649080 12496 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.569837 12504 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:47.636844  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.660036  9826 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.666106  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:47.668634  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:47.692997  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.677151  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 676) took 1410 ms. Trace:
I20260709 13:04:47.679883  9264 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:04:47.694659  9266 rpcz_store.cc:276] 0709 13:04:46.266465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.266633 (+   168us) service_pool.cc:224] Handling call
0709 13:04:47.631135 (+1364502us) spinlock_profiling.cc:243] Waited 9.37 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:47.677074 (+ 45939us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21543296}
I20260709 13:04:47.695456  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:47.696280  9264 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:47.590726 12018 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.700230  9190 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:04:47.699369 12018 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.657721 12442 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:47.707218  9264 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:47.666448 11805 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.711787 12504 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.711982 11805 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.676594 12277 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:47.742694  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 52043.6us
W20260709 13:04:47.746544  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 86601.8us
W20260709 13:04:47.740401 12508 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:04:47.714272 12442 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.767150  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.767874  9271 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:04:47.773268  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.693526 12474 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.753717 12348 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.775802  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:47.767979 12442 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:47.778774 12506 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:47.779325 12506 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.781476 12506 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.752780 12500 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:47.787652  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:47.755383 12512 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:47.791862  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47669.4us
I20260709 13:04:47.792619  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:47.805604 12470 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.808238 12510 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:47.804014  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:47.822712  9271 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:47.802867 12376 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.704244 12496 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.816248 12113 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:47.839735  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:47.839835  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 66673.2us
I20260709 13:04:47.790733 12348 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.822242 12470 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.844866 12523 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:47.837384 12248 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
W20260709 13:04:47.851624 12514 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:04:47.854411  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:47.792315 12500 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.874472  9268 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:47.866533  9271 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:47.883394  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.864351 12113 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:47.876683 12497 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:47.880698  9162 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:47.873188 12523 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.889467  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.890813 12122 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:47.892010 12472 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.893602 12369 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:47.894259 12348 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.903671  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.793988 12468 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.887923 12523 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:47.916272  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:47.888166 12423 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:47.920804 11805 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.922945 12474 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.891363 12122 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.881116 12069 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:47.910214 12517 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:04:47.902631 12374 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:47.925784 12442 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:47.894912 12525 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:47.918656 12497 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:47.916698 12436 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:04:47.937167 12496 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.937803 12500 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.923375 12423 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:47.941223  9854 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:47.955233  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1979) took 1402 ms. Trace:
I20260709 13:04:47.955895  9194 rpcz_store.cc:276] 0709 13:04:46.552881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.555537 (+  2656us) service_pool.cc:224] Handling call
0709 13:04:46.574951 (+ 19414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.096853 (+521902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.954875 (+858022us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400398d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:47.955124 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1198459904}
I20260709 13:04:47.928135 12113 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:47.947273 12423 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:47.964221 12122 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.956635  9019 catalog_manager.cc:5697] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
W20260709 13:04:47.971762  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:47.972489 12360 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:47.974524 12071 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:47.974676 12440 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:47.975049 12222 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:47.974860 12360 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.975287 12440 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:47.975579 12222 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.976194  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.976800  9192 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:47.976962 12360 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.977604 12222 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.977716  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.977905 12442 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.978562  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.978322 12071 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.978922  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.979146  9189 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:47.979125 12069 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.980093  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.980561 12071 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:47.977360 12440 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.980050  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.984170 12366 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:47.985142  9120 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:04:47.979535  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:47.978343  9118 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:04:47.981926  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:47.986660  9194 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
W20260709 13:04:47.985450 12492 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.432s	user 0.000s	sys 0.001s
I20260709 13:04:47.993209 12366 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:47.995783  9828 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:48.005389  9828 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:48.008271  9828 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:48.014655 12423 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:47.994948 12509 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:48.018502 12500 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:48.016582 12069 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:04:48.019659 12069 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:48.020157 12069 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:48.021286 12472 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:48.021104 12069 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.022454 12069 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:48.023952 12458 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:48.024927 12523 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.012511 12366 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:48.018683  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:48.015933 12371 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:48.041827 12371 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:48.040637 12458 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.040678 12423 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:48.042749 12458 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.044083 12371 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:48.045045 12458 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:48.045414 12366 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:48.045820 12366 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:48.046623 12366 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.047863 12523 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:48.048128 12366 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:48.048725 12472 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.049232 12472 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.051204 12472 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:48.052476  9118 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:48.053844  9121 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:48.054862  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:48.055467  9116 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:48.056183  9828 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:48.056667 11805 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:48.057379  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:48.058132 12458 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:48.058802 12458 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:48.059135 12458 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.059486 12458 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:48.066035  9828 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:48.067314  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:48.067417  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:48.068501  9120 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:48.129431  9117 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:04:48.071913 12167 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:48.146948  9118 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:04:48.438302  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.281085 12468 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:48.438076 12422 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:48.528417 12468 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:48.111985 12366 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:48.529186 12468 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:48.468230 12500 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.530189 12500 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.528798 12422 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.531407  9115 raft_consensus.cc:2955] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:48.532820 12500 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:48.529358  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:48.534067 12422 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:48.436685 12440 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:48.529232 12366 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:48.536350 12167 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.318029 12458 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.537809  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:48.538304  9858 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:48.538863  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:48.556125  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:48.568405  9189 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:48.571890  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:48.569169  9194 raft_consensus.cc:2955] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:04:48.578472  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.579732  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:48.585913  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.586349  9121 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:48.586128  9196 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:48.228720 12474 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:48.555683 12472 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.589819  9195 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:48.588487  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:48.573907 12366 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:04:48.580152 12537 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.361s	user 0.000s	sys 0.001s
I20260709 13:04:48.594236  9860 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:04:48.594592  9860 pending_rounds.cc:85] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 5
I20260709 13:04:48.594832  9860 pending_rounds.cc:107] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:04:48.595070  9860 raft_consensus.cc:2889] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:48.598367  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.598990  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:48.601297  9860 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:48.602450  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:48.593061  9019 catalog_manager.cc:5697] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 1 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:04:48.589895 12500 consensus_queue.cc:1059] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:04:48.573745 12440 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:48.605381  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:48.609006  9861 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:48.609760  9860 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:48.615576  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.616219  9857 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:48.593688 12472 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.593775 12474 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.627106  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 1991) took 2040 ms. Trace:
I20260709 13:04:48.627327  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.628199  9860 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:48.627094  9861 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:48.629748  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.627600  9197 rpcz_store.cc:276] 0709 13:04:46.586857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.615205 (+ 28348us) service_pool.cc:224] Handling call
0709 13:04:46.652809 (+ 37604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:46.749394 (+ 96585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.626846 (+1877452us) spinlock_profiling.cc:243] Waited 95.9 ms on lock 0x7b54001d4f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:48.627039 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220508928}
I20260709 13:04:48.661242  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.661918  9861 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:48.664371  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:48.665169  9857 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:04:48.629683 12369 raft_consensus.cc:2764] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:04:48.656114 12422 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.677881 12458 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:48.678731  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 910) took 1228 ms. Trace:
I20260709 13:04:48.679778  9854 rpcz_store.cc:276] 0709 13:04:47.445152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.445615 (+   463us) service_pool.cc:224] Handling call
0709 13:04:48.673487 (+1227872us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b540064ffd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:48.673637 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1019646720}
I20260709 13:04:48.682881  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.683769  9861 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:04:48.701539  9857 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:48.700868 12108 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.705821 12108 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:48.708503 12108 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:48.710296 12532 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.774s	user 0.006s	sys 0.022s
W20260709 13:04:48.717077  9114 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 2108) took 2106 ms (client timeout 2045 ms). Trace:
W20260709 13:04:48.717700  9114 rpcz_store.cc:269] 0709 13:04:46.611308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.611477 (+   169us) service_pool.cc:224] Handling call
0709 13:04:48.716817 (+2105340us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54001e3f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:48.716976 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645541760}
I20260709 13:04:48.719214  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 714) took 1059 ms. Trace:
I20260709 13:04:48.728585  9269 rpcz_store.cc:276] 0709 13:04:47.660068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.693901 (+ 33833us) service_pool.cc:224] Handling call
0709 13:04:47.741816 (+ 47915us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.772355 (+ 30539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.772384 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:47.785145 (+ 12761us) log.cc:844] Serialized 26 byte log entry
0709 13:04:47.874175 (+ 89030us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:47.874620 (+   445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:47.874703 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:48.709540 (+834837us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:48.712782 (+  3242us) raft_consensus.cc:1685] finished
0709 13:04:48.712822 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:48.719133 (+  6311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26168,"raft.queue_time_us":30224,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":12712,"spinlock_wait_cycles":2490112,"thread_start_us":64468,"threads_started":1,"wal-append.queue_time_us":93000}
W20260709 13:04:48.701426 12485 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.333s	user 0.000s	sys 0.014s
I20260709 13:04:48.745558 11517 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:48.687839 12422 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:48.746022 11517 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:48.743464 12167 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:48.769273 12539 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:04:48.788689 12372 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:48.789489 12372 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.769435 12076 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:48.781544 12472 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:48.793270 12069 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.770s	user 0.014s	sys 0.000s
I20260709 13:04:48.785627 12440 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:48.812495 12167 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:48.816970 12372 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:48.853413  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 264988us
I20260709 13:04:48.830278 12076 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.776917 11517 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:48.848752 12472 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:48.857452 12538 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:04:48.851686 12474 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:48.875250  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:48.877682  9195 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:48.856964 12167 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:48.873663 11517 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:48.882992  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 556) took 1309 ms. Trace:
I20260709 13:04:48.883361  9190 rpcz_store.cc:276] 0709 13:04:47.573434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.580158 (+  6724us) service_pool.cc:224] Handling call
0709 13:04:47.593016 (+ 12858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.700750 (+107734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.882744 (+1181994us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54001d5954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:48.882931 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245514624}
W20260709 13:04:48.886008  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:48.873962 12472 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:48.878706 12440 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:04:48.900645  9856 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:04:48.901646  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.902524  9859 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:48.903970 12422 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:48.904145  9855 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:48.905838  9854 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:04:48.907840  9123 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:04:48.910904  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:48.912667  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.913751  9861 raft_consensus.cc:2410] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:48.914932  9857 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:48.915091  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:48.915778  9856 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:48.916373  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.920024  9855 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:48.917075  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.921687  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:48.922621  9114 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:48.916437  9859 raft_consensus.cc:2955] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 3 to 6, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:48.924911  9855 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:48.926614  9855 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:48.932948  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2267) took 1115 ms. Trace:
I20260709 13:04:48.933158  9857 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:48.933482  9268 rpcz_store.cc:276] 0709 13:04:47.817731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.817936 (+   205us) service_pool.cc:224] Handling call
0709 13:04:47.818311 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.879035 (+ 60724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.932635 (+1053600us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b54004448d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:48.932877 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124051328}
W20260709 13:04:48.937896  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 47080.5us
I20260709 13:04:48.910435 12543 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:48.932331 12076 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:48.941896 12541 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:04:48.958873  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:48.960659  9854 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:04:48.921945 12474 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
W20260709 13:04:48.967442  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 57463.5us
I20260709 13:04:48.966310  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:48.968502  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:48.970667  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:48.979279  9856 raft_consensus.cc:2410] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:48.975869 12076 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:04:48.987556 12167 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:48.962455 12511 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:48.961720 12493 raft_consensus.cc:3055] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:48.998310 12493 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:48.999405 12493 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.000856 12493 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:49.009626  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:49.011881  9854 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:48.959048 12422 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:04:49.012917  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.013190  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.018787  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.013943  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:49.014420  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:49.014621  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.020819  9189 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:49.015461  9192 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:04:49.022289  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:48.995673 12440 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:49.019568  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.024992  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:49.022967  9089 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.028339  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.028455  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:49.029654  9089 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.030422  9190 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:48.944717 12543 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.027505  9198 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:49.005733 12558 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:49.028398  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.034344  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.035018  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.026320  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.007579 12167 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:49.006659 12386 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.007606 12540 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:49.041090  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:49.044548  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.016857 12269 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:49.056226  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.058188  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:49.059023  9857 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:49.025753 12227 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.065469  9123 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:04:49.032799 12550 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:49.056529  9122 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:49.073016  9192 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:04:49.106580  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.032889 12558 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:49.006856 12511 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.041174 12076 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:49.048307 12386 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.131551 12559 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.131582 12076 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:49.132474 12560 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:04:49.132272 12559 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.036352 12482 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:04:49.134297 12372 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.134549 12543 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:49.133230 12076 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.134859 12372 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.135147 12543 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:49.137188  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:49.129956 12558 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.055817 12540 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.142184  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:49.143046  9121 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:49.143656  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:49.036415 12440 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:49.127692  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 89701.5us
I20260709 13:04:49.127884  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.033056 12382 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:04:49.148048 12543 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:49.148495 12543 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:49.149245 12543 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.150141 12543 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:49.151815  9190 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:49.155951  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:49.163720  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:49.144098 12546 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:04:49.162361  9195 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:49.125077 12550 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:04:49.146329 12551 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:04:49.126587 12227 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.169888  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:49.176932 12553 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:04:49.169842 12423 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:49.173115 12108 raft_consensus.cc:2764] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:04:49.199585 12423 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:49.200498 12423 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.201694 12423 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:49.203337 12497 raft_consensus.cc:697] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:49.147007 12472 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:49.068985 12269 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.205451 12511 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:49.205695 12269 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:49.180174 11805 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:49.181059  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.208695  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.208428 12440 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.211256 12436 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.211782 12436 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.212594 12152 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:49.213292  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2259) took 1666 ms. Trace:
I20260709 13:04:49.213218 12152 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.214074 12436 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:49.215305 12291 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:49.174228 12550 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:49.215701  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.215843 12291 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:49.216594  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:49.197067  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 1.28082e+06us
I20260709 13:04:49.217903 12291 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:49.217767 12528 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:49.219286 12496 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:49.197662 12389 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:49.219900 12496 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.220559 12494 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.220953 12498 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:49.221100 12494 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.175966  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.223476  9121 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:49.229157  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.230384  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:49.230779  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.231500  9122 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
W20260709 13:04:49.239005  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 83190.1us
W20260709 13:04:49.250111 12468 consensus_meta.cc:287] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.720s	user 0.007s	sys 0.003s
I20260709 13:04:49.250844 12468 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.252338  9858 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:49.213810  9272 rpcz_store.cc:276] 0709 13:04:47.546455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.546607 (+   152us) service_pool.cc:224] Handling call
0709 13:04:47.546924 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:47.639361 (+ 92437us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:47.770268 (+130907us) log.cc:844] Serialized 14 byte log entry
0709 13:04:49.211348 (+1441080us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:49.211426 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:49.211476 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.213001 (+  1525us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b54000fb4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:49.213057 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.213216 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1273498,"prepare.queue_time_us":135843,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":40,"raft.queue_time_us":1216002,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":212067584,"thread_start_us":289569,"threads_started":2,"wal-append.queue_time_us":269586,"wal-append.run_cpu_time_us":1182,"wal-append.run_wall_time_us":1008537}
I20260709 13:04:49.257079  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:49.257820  9264 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:49.259927  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.268682  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:49.269594  9272 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.271706  9164 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:49.273882  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.221843 12496 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:49.275923  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 130408us
I20260709 13:04:49.282454  9264 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:04:49.286002  9265 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:49.176267 12540 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:49.267724 12468 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:49.289758  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 78676.2us
I20260709 13:04:49.219924 12282 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:49.190873 12372 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:49.181725 12386 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:49.295109  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 76557.8us
I20260709 13:04:49.265290 12505 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:49.301999  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.276785 12510 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:49.265399 11805 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:04:49.265813 12269 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:04:49.291982 12389 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:49.379518  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 155097us
I20260709 13:04:49.377840  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.405855  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.371863  9123 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:04:49.376015  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.278707 12511 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:04:49.440925  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 65730.1us
I20260709 13:04:49.304524  9121 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:49.409893  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.443177  9118 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:49.444507  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:49.417900 12497 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.422791 12494 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:49.425529  9263 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:49.437637  9272 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:49.307278  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:49.452502  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.305263  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:49.443609  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:49.468461  9121 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:49.454763  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:49.470191  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.454355  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:49.372578 12552 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:49.470326  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:49.511868  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.541488  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.472527  9268 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:04:49.549508  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 79392us
I20260709 13:04:49.471731  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:49.483289  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 178107us
I20260709 13:04:49.296851 12382 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.168s
I20260709 13:04:49.296566 12167 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:49.578020  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 244549us
I20260709 13:04:49.301426 12152 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:49.369558 12559 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:49.445124 11805 consensus_queue.cc:1243] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:04:49.604801  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 129690us
I20260709 13:04:49.564009 12356 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:49.565498 12076 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:49.614583 12356 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:49.614854 12076 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.615599 12356 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.616107  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:49.616660  9117 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:49.617672 12076 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:49.298213 12416 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.320s
I20260709 13:04:49.623118  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:49.624173 12545 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:49.376359 12526 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.255s
W20260709 13:04:49.627693 12564 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.218s	user 0.000s	sys 0.000s
W20260709 13:04:49.629482 12570 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
I20260709 13:04:49.304131 12540 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.262s
I20260709 13:04:49.402073 12291 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.177s
I20260709 13:04:49.636056  9121 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:49.637164  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:49.639176  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.464282 11801 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:49.535907 12277 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:49.337445 12505 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.662015  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 963) took 1064 ms. Trace:
I20260709 13:04:49.580507 12355 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:49.659327  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 364857us
I20260709 13:04:49.649997 12152 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:04:49.567962 12371 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:49.662410  9858 rpcz_store.cc:276] 0709 13:04:48.584103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.584923 (+   820us) service_pool.cc:224] Handling call
0709 13:04:48.585191 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.252807 (+667616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.625996 (+373189us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54006013d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:49.648557 (+ 22561us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1533802368}
I20260709 13:04:49.337100 12498 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.274s
I20260709 13:04:49.579118 12494 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:49.336752 12386 consensus_queue.cc:1059] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.296s
I20260709 13:04:49.659181  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.302070 12282 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:49.610378 12366 consensus_meta.cc:287] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.009s	user 0.011s	sys 0.002s
W20260709 13:04:49.742993  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 93545.6us
W20260709 13:04:49.666818 12493 consensus_meta.cc:287] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.003s
W20260709 13:04:49.766211  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 197169us
W20260709 13:04:49.742731  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 104009us
I20260709 13:04:49.737993  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.769192  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.455423 12510 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:49.782320  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 118054us
I20260709 13:04:49.666463 11801 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.837004  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.733603 12277 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:49.736129 12558 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.000s
I20260709 13:04:49.464551 12389 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:49.659262 11517 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.762s	user 0.007s	sys 0.004s
I20260709 13:04:49.735183 12152 consensus_queue.cc:1243] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:04:49.649104 12568 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:49.768234 12543 consensus_meta.cc:287] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.618s	user 0.008s	sys 0.002s
I20260709 13:04:49.658578 12474 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 13:04:49.826545  9117 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:04:49.830101  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 180824us
I20260709 13:04:49.738617 12108 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:04:49.658740 12566 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:49.835150  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:49.832450  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 63399.1us
I20260709 13:04:49.842063  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.898522  9192 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:49.842509  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.758096 12355 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.843801  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.667014 12526 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:49.845062  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.845723  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.831372  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:49.859449  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.805503 12494 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:49.887588  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.831235 12572 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.910650  9197 raft_consensus.cc:2410] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:49.861967  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.892692  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 2125) took 1354 ms. Trace:
I20260709 13:04:49.911381  9117 rpcz_store.cc:276] 0709 13:04:48.538645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.615829 (+1077184us) service_pool.cc:224] Handling call
0709 13:04:49.892635 (+276806us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:49.836508 12076 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:49.893151  9122 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:49.847568 12559 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:49.843597  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:49.825266 12371 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.903008  9191 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:04:49.905324 12577 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
W20260709 13:04:49.907342  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 76040.4us
W20260709 13:04:49.907538  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 62167.9us
I20260709 13:04:49.906983  9189 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:49.923794  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.904902 12583 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:04:49.912986  9121 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:49.925891 12511 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:49.861259 12376 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:04:49.882377 12558 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.917111  9190 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:49.825624 12167 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.923697  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:49.924589  9117 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:49.909693  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:49.833671 12366 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.922463  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:49.847314 12529 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 13:04:49.929441  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:49.896665 12568 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.910996 12494 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:49.905994 12526 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:49.950395 12269 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:49.950572 12526 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.916055 12076 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:49.951982 12076 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:49.911713 12543 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:49.935261  9825 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.957307  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.916224 12572 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:49.936785 12581 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 13:04:49.937215 12069 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:49.966677  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 398966us
I20260709 13:04:49.937875 12458 raft_consensus.cc:3055] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:49.938637 12558 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:49.938674  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:49.928908 12576 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 13:04:49.948069  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 113049us
I20260709 13:04:49.972050  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:49.947410  9161 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:49.960258  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 50664.8us
I20260709 13:04:48.964309  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:49.961829  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:49.973371  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:49.975931  9860 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:49.922768 12559 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:49.974480  9162 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:49.945067 12277 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:49.980001  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.991706  9267 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:04:49.907042 12566 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.943688 12227 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:49.970502 12108 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:04:49.981191  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 157232us
I20260709 13:04:50.022775 12108 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:50.022883 12500 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:04:50.023267  9087 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.110s (SENT) [suppressed 27 similar messages]
I20260709 13:04:50.023388 12500 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:50.023861 12108 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.024422 12500 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:49.980319  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:04:50.025772  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45532.4us
I20260709 13:04:49.993208  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:49.992177  9161 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:49.960480 12580 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:49.981091  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1802) took 2287 ms. Trace:
I20260709 13:04:49.964972 12422 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:49.930796 12573 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
W20260709 13:04:50.007246 12556 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.856s	user 0.004s	sys 0.000s
I20260709 13:04:50.010429 12526 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.010553  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.011440  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.148015  9264 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:50.012207  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.003458 12113 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:50.004213 12277 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:50.022092 12550 consensus_meta.cc:287] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.806s	user 0.002s	sys 0.007s
I20260709 13:04:49.973194 12458 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:49.973088 12545 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:50.029788  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:50.163487  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:49.976969 12586 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.055s	user 0.001s	sys 0.001s
I20260709 13:04:50.003625 12376 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:50.026903  9089 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:50.088946  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:50.089519  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 96330us
I20260709 13:04:50.090397  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2115) took 2024 ms. Trace:
I20260709 13:04:50.080199  9266 rpcz_store.cc:276] 0709 13:04:47.693975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.695976 (+  2001us) service_pool.cc:224] Handling call
0709 13:04:47.696368 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:47.706419 (+ 10051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:47.706463 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.988731 (+1282268us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b2c00112948. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:48.989196 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.287253 (+298057us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:49.287482 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.287726 (+   244us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.973465 (+685739us) raft_consensus.cc:1685] finished
0709 13:04:49.973485 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:49.980995 (+  7510us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21954,"raft.queue_time_us":10115,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":26034,"spinlock_wait_cycles":16344320,"thread_start_us":297015,"threads_started":1,"wal-append.queue_time_us":347022}
I20260709 13:04:50.148711  9269 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:49.995390 12382 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:50.151430  9268 raft_consensus.cc:2410] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:04:50.153728  9117 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:50.157984  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:50.171159  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 144311us
I20260709 13:04:50.192364  9264 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:49.990777 12386 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:50.158229 12277 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:50.078054 12558 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:50.161662 12423 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:49.967101 12474 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:50.169667  9263 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:50.086612 12510 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:50.198665  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:04:50.212221  9089 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:50.151371 12113 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:50.005054 12587 consensus_queue.cc:1059] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.187s
I20260709 13:04:50.081355 12440 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:04:50.163190 12416 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:50.167971 12500 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:50.190131  9116 rpcz_store.cc:276] 0709 13:04:48.065946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.069527 (+  3581us) service_pool.cc:224] Handling call
0709 13:04:48.069731 (+   204us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.076467 (+  6736us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:48.076512 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.130871 (+ 54359us) log.cc:844] Serialized 26 byte log entry
0709 13:04:49.201685 (+1070814us) spinlock_profiling.cc:243] Waited 977 us on lock 0x7b18000755e0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:04:49.339163 (+137478us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:49.339228 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.339335 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.029950 (+690615us) raft_consensus.cc:1685] finished
0709 13:04:50.029976 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.090319 (+ 60343us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":212625,"raft.queue_time_us":8742,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":35950,"spinlock_wait_cycles":2247424,"thread_start_us":125897,"threads_started":1,"wal-append.queue_time_us":1226705}
I20260709 13:04:50.209354 12584 raft_consensus.cc:3055] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:50.242136 12510 raft_consensus.cc:3055] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:50.256116 12584 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:50.256268 12510 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:50.257268 12552 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:50.258069 12552 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:50.222481 12382 raft_consensus.cc:697] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:50.214110 12113 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:50.210619 12376 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:50.250589 12559 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:50.228211 12416 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.202185 12277 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:50.280656 12568 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:50.281883 12277 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:50.279992  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 286897us
I20260709 13:04:50.283913  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 949) took 1383 ms. Trace:
I20260709 13:04:50.149920 12580 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.257067 12584 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.197146 12545 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:50.162230 12550 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.209103 12474 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:50.244931 12579 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.823s	user 0.002s	sys 0.000s
I20260709 13:04:50.271876  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.359637  9267 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:50.228703 12558 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:50.360873 12382 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:50.361719 12558 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:50.362375 12558 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.362639 12258 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:50.273233  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.366669  9017 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42488 (request call id 239) took 23483 ms (client timeout 15000 ms). Trace:
I20260709 13:04:50.277872  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:50.372321  9019 catalog_manager.cc:5697] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 4, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:04:50.293079  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:50.284452  9862 rpcz_store.cc:276] 0709 13:04:48.900693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.902463 (+  1770us) service_pool.cc:224] Handling call
0709 13:04:48.902780 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:48.903168 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:48.903196 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:48.903720 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:04:48.966856 (+ 63136us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:48.966896 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:48.966971 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:49.927575 (+960604us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:50.267370 (+339795us) raft_consensus.cc:1685] finished
0709 13:04:50.267397 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.283829 (+ 16432us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1584,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"spinlock_wait_cycles":14135552,"thread_start_us":39343,"threads_started":1,"wal-append.queue_time_us":138458}
I20260709 13:04:50.304112  9162 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:50.310685 12572 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:50.311277 12603 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:50.317344 12550 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:50.317766 12474 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.319060  9114 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:50.410594  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.411305  9855 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:50.318993  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2277) took 1100 ms. Trace:
I20260709 13:04:50.326006 12566 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:50.337476 12376 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.422607  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:50.338888 12505 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:50.257071 12510 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:50.425377 12510 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.287031 12416 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:50.344756 12545 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.237862 12458 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.430692 12458 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.430953 12423 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:50.305346 12559 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:50.354573  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.431720 12559 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.412348  9270 rpcz_store.cc:276] 0709 13:04:49.218740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.222008 (+  3268us) service_pool.cc:224] Handling call
0709 13:04:49.276791 (+ 54783us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:49.299679 (+ 22888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.299706 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:49.309051 (+  9345us) log.cc:844] Serialized 582 byte log entry
0709 13:04:49.649883 (+340832us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:49.649939 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:49.650014 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.318760 (+668746us) raft_consensus.cc:1685] finished
0709 13:04:50.318793 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.318933 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7195,"raft.queue_time_us":8477,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":33585,"thread_start_us":215393,"threads_started":1,"wal-append.queue_time_us":457329}
W20260709 13:04:50.368095  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.432114 12559 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:50.436031 12572 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:50.436609 12572 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.438961 12572 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:50.294674 12076 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.441488 12076 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:50.443492 12580 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:50.298627 12605 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:50.445094 12584 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:50.251663 12575 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:50.287627  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:50.375079  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 77853.5us
W20260709 13:04:50.380228  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.406113  9862 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:50.404654  9236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.021s	sys 0.008s
I20260709 13:04:50.414036  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:50.460994  9236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.021s	sys 0.008s
I20260709 13:04:50.460737  9162 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:50.461726  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:50.461933  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 249783us
I20260709 13:04:50.463389  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:50.432132  9265 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:50.464694  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.415158  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.416836  9019 catalog_manager.cc:5697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 4, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:04:50.417778  9826 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:50.420411  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:50.421420  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:50.466631  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.465684  9859 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:04:50.472553  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:50.483819  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.484736  9853 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:50.486126  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 68422.8us
I20260709 13:04:50.483687  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.487825  9855 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:50.484848  9271 raft_consensus.cc:2410] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:04:50.497540  9192 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:04:50.497762  9238 connection.cc:256] RPC call timeout handler was delayed by -1.2784546219999697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
I20260709 13:04:50.498907  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:50.499697  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.496515  9019 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 5 to 6, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 6 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.536075  9856 raft_consensus.cc:1217] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[3.5-7.7]   Dedup: 3.5->[5.6-7.7]
I20260709 13:04:50.536693  9856 pending_rounds.cc:85] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 5
I20260709 13:04:50.536996  9856 pending_rounds.cc:107] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:04:50.537297  9856 raft_consensus.cc:2889] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:50.419334 12482 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:50.540921  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 75569us
I20260709 13:04:50.543522  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:50.552781  9189 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:50.560159 12600 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.207s	user 0.003s	sys 0.000s
I20260709 13:04:50.562520  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2293) took 1136 ms. Trace:
I20260709 13:04:50.564174  9189 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:04:50.565407 12366 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:50.543919  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:50.548815  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.570259  9263 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:50.550139  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.551007  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.572106  9266 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:50.572568  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.572904  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.563030  9272 rpcz_store.cc:276] 0709 13:04:49.426350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.467922 (+ 41572us) service_pool.cc:224] Handling call
0709 13:04:49.468165 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.890378 (+422213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:49.890428 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:49.890480 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:49.890525 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.562250 (+671725us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b5400302fd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:50.562322 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:50.562468 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":965772544}
I20260709 13:04:50.576092 11801 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:50.577644 12282 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:50.578624 12578 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 1.154s	user 0.004s	sys 0.000s
I20260709 13:04:50.592352 11805 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:50.592986 11805 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:50.594120 11805 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.599031 12355 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:50.606683  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:50.607772  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:50.607991  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:50.608413  9114 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:50.609220  9123 raft_consensus.cc:1217] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-6.4]   Dedup: 2.2->[4.3-6.4]
I20260709 13:04:50.609684  9123 pending_rounds.cc:85] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 2
I20260709 13:04:50.609892  9123 pending_rounds.cc:107] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:04:50.610100  9123 raft_consensus.cc:2889] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:50.611028  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:50.612582  9117 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:50.615794 12389 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:50.618548 12595 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.607s	user 0.000s	sys 0.002s
I20260709 13:04:50.619925  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 973) took 1988 ms. Trace:
I20260709 13:04:50.620496  9860 rpcz_store.cc:276] 0709 13:04:48.631504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:48.631789 (+   285us) service_pool.cc:224] Handling call
0709 13:04:48.634508 (+  2719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:49.976746 (+1342238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.619578 (+642832us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400716454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:50.619844 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3084464512}
I20260709 13:04:50.621970  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:50.622778  9119 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:50.587293 12371 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:50.571475  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:50.632273  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.633641  9235 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:50.636008 11805 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:50.418231 12568 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:50.646697  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 183362us
I20260709 13:04:50.647258  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:50.647918  9267 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:50.652443  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:50.652928  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.647444  9089 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:50.658521  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.670140  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.496963 12566 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.457546 12575 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:50.532364 12567 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:50.677700  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.483536 12493 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:50.678478  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.673988  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.673421  9190 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:50.410473 12603 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.533711 12505 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.684079  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.701498  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:50.656616 12227 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.701351  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:50.551394 12565 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:50.684274  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:50.646427 12572 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:50.648733  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.755144  9855 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:50.644132 12605 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:50.757591  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 109528us
I20260709 13:04:50.757135  9019 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 7 to 8, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 8 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
W20260709 13:04:50.701869 12497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.014s	sys 0.017s
W20260709 13:04:50.771175  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 137666us
I20260709 13:04:50.691942 12567 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.698907 12552 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.532613 12416 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:50.769938  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.536245 12113 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:50.786761  9019 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.779397  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:50.779646  9116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.026s	sys 0.085s
W20260709 13:04:50.791771  9116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.027s	sys 0.085s
I20260709 13:04:50.718850 12371 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:50.786099  9857 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:50.706961 12069 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:50.766445  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:50.780220  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:50.793074  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:50.793435  9115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.053s	sys 0.071s
I20260709 13:04:50.753317 12277 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:50.680135 12568 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.779771  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:50.797662  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 138788us
I20260709 13:04:50.777928 12567 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:50.799634  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:50.801019  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:50.821311  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.671479 12610 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.134s
W20260709 13:04:50.789551 12573 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.012s	sys 0.028s
I20260709 13:04:50.692341 12566 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:50.809881  9115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.053s	sys 0.071s
I20260709 13:04:50.779582 12426 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:50.811270  9197 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:50.756618 12227 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.752918 12505 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:50.782092 12598 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.137s
I20260709 13:04:50.709125 12493 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:50.841934  9196 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:04:50.787937  9267 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:50.847925  9195 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:04:51.075546  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 576710us
I20260709 13:04:50.958431  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:50.896255  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:51.076418  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:51.076495  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:51.076957  9265 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
W20260709 13:04:50.894387  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52763.2us
I20260709 13:04:50.959446  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:51.079523  9235 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:50.959061  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:50.756374 12605 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:50.976488  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:50.375054  9017 rpcz_store.cc:269] 0709 13:04:26.883447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:26.885215 (+  1768us) service_pool.cc:224] Handling call
0709 13:04:50.366437 (+23481222us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:37.466259 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:37.466921 (+   662us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:37.466972 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:04:37.466994 (+    22us) tablet.cc:662] Decoding operations
0709 13:04:37.639944 (+172950us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:37.640022 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:37.640054 (+    32us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:04:37.774946 (+134892us) tablet.cc:701] Row locks acquired
0709 13:04:37.774997 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:04:37.775190 (+   193us) write_op.cc:270] Start()
0709 13:04:37.775337 (+   147us) write_op.cc:276] Timestamp: P: 1783602277775114 usec, L: 0
0709 13:04:37.775354 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:04:37.776061 (+   707us) log.cc:844] Serialized 4804 byte log entry
0709 13:04:38.432308 (+656247us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b180038a6a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:39.110352 (+678044us) op_driver.cc:464] REPLICATION: finished
0709 13:04:39.767551 (+657199us) write_op.cc:301] APPLY: starting
0709 13:04:39.783690 (+ 16139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:40.116725 (+333035us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:40.128347 (+ 11622us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:40.163761 (+ 35414us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:40.248341 (+ 84580us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:04:40.248392 (+    51us) write_op.cc:312] APPLY: finished
0709 13:04:40.248766 (+   374us) log.cc:844] Serialized 89 byte log entry
0709 13:04:40.364060 (+115294us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:40.364241 (+   181us) write_op.cc:454] Released schema lock
0709 13:04:40.364323 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:40.418125 (+ 53802us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:40.422563 (+  4438us) write_op.cc:454] Released schema lock
0709 13:04:42.862186 (+2439623us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b180038a6a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
Metrics: {"mutex_wait_us":392088,"reported_tablets":31,"spinlock_wait_cycles":8448,"child_traces":[["op",{"apply.queue_time_us":625692,"apply.run_cpu_time_us":14132,"apply.run_wall_time_us":635810,"num_ops":12,"prepare.queue_time_us":393844,"prepare.run_cpu_time_us":45084,"prepare.run_wall_time_us":739042,"raft.queue_time_us":371158,"raft.run_cpu_time_us":61392,"raft.run_wall_time_us":528366,"replication_time_us":1310521,"spinlock_wait_cycles":37614720,"thread_start_us":1416931,"threads_started":5,"wal-append.queue_time_us":666713,"wal-append.run_cpu_time_us":114182,"wal-append.run_wall_time_us":2728200}]]}
I20260709 13:04:51.077831  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:51.082393  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 381128us
I20260709 13:04:50.958791  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:51.075143  9859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.016s	sys 0.008s
W20260709 13:04:51.110705  9859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.016s	sys 0.009s
I20260709 13:04:51.096995  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:50.960055  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:51.117717  9270 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
W20260709 13:04:51.118774  9856 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.008s	sys 0.015s
W20260709 13:04:51.119525  9856 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.008s	sys 0.015s
I20260709 13:04:50.928192  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:51.035699  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:50.896265  9272 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:04:51.122865  9272 raft_consensus.cc:606] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:50.894675  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:51.124642  9855 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:51.128402  9263 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:50.987275  9019 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 5 to 6, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 6 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:51.128580  9192 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.009s	sys 0.023s
W20260709 13:04:51.129138  9192 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.010s	sys 0.023s
I20260709 13:04:51.127454  9264 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:51.084996  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:51.134905  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:50.953694 12376 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:04:51.139290 12494 consensus_meta.cc:287] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.182s	user 0.008s	sys 0.002s
I20260709 13:04:51.139787 12356 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:51.098662 12291 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.140552 12416 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:51.141276 12227 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:50.917484 12366 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:50.812319 12498 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:51.143878  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:51.078989 12573 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.031s	sys 0.050s
I20260709 13:04:51.202564  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:51.144807  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:50.809031 12277 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:51.148977  9194 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:51.148825  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:51.077509 12426 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:51.112453 12382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.012s	sys 0.025s
I20260709 13:04:50.895308 12269 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.178654 12545 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:51.160640  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:51.114400 12510 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:51.224835  9270 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:51.204314 12498 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:51.221426 12382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.012s	sys 0.025s
I20260709 13:04:51.219977 12277 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:50.893618 12593 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:51.115491 12606 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.192040  9263 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:50.807087 12371 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.196508  9119 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:04:51.177122 12291 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.204205  9264 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:51.256489 12611 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.485s	user 0.000s	sys 0.001s
I20260709 13:04:51.245847 12113 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:51.203500  9272 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:51.215723  9267 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:51.196959  9019 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 6, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 6 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.923061 12497 consensus_queue.cc:1059] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.257s
I20260709 13:04:50.841364 12069 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.221014 12426 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:51.155084 12494 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.168388 12356 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.230804 12269 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.152269  9857 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 3.
I20260709 13:04:51.220625 12376 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:51.127372 12152 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:51.257660  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 179894us
I20260709 13:04:51.299989  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:51.301435  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:51.302232  9855 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:51.302589  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:51.258749 12606 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:51.259449 12619 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:04:51.231051 12510 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:51.310268  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:51.224226 12574 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:51.233474 12498 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:51.226145 12612 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:51.223682 12626 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:51.230139 12366 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:04:51.266798 12526 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.113s	user 0.011s	sys 0.000s
I20260709 13:04:51.293130  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:51.294464  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:51.288996 12620 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:04:51.308462  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:51.313982 12440 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.009s	sys 0.026s
I20260709 13:04:51.393604 12610 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:51.394590  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:51.406387  9272 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
W20260709 13:04:51.427572  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 174495us
I20260709 13:04:51.380138  9019 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 1 to 4, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 4 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.305545 12568 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:51.439783  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:51.338973  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2192) took 1313 ms. Trace:
W20260709 13:04:51.445974 12608 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.487s	user 0.000s	sys 0.001s
I20260709 13:04:51.422745 12626 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:51.448347 12618 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:51.448323 12626 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:51.453459 12621 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:04:51.455211  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2004) took 1041 ms. Trace:
W20260709 13:04:51.444133  9164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.003s
W20260709 13:04:51.459918  9164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.008s	sys 0.003s
W20260709 13:04:51.395211 12440 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.010s	sys 0.027s
I20260709 13:04:51.317495 12510 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:51.388599 12566 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:51.305007 12543 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:51.396157  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:51.493430  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:51.398876  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:51.314677  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:51.455615  9191 rpcz_store.cc:276] 0709 13:04:50.413950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.414348 (+   398us) service_pool.cc:224] Handling call
0709 13:04:50.414707 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.423072 (+  8365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.423102 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.423575 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.702153 (+278578us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:50.747983 (+ 45830us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.780084 (+ 32101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.780153 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.454964 (+674811us) raft_consensus.cc:1685] finished
0709 13:04:51.454985 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.455137 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":190364,"prepare.queue_time_us":45596,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":40286,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":16501,"thread_start_us":278179,"threads_started":1,"wal-append.queue_time_us":829636}
I20260709 13:04:51.305301 12376 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:51.435935  9019 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 5 to 6, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 6 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.356505  9269 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:51.494621  9857 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:51.462250  9264 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
W20260709 13:04:51.498876  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 100091us
W20260709 13:04:51.460640  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 667592us
I20260709 13:04:51.441485 12505 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:51.541790  9019 catalog_manager.cc:5697] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:04:51.444906  9122 rpcz_store.cc:276] 0709 13:04:50.025245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.197965 (+172720us) service_pool.cc:224] Handling call
0709 13:04:50.198651 (+   686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.199076 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.199113 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.200400 (+  1287us) log.cc:844] Serialized 499 byte log entry
0709 13:04:50.485387 (+284987us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:50.485424 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.485492 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.338707 (+853215us) raft_consensus.cc:1685] finished
0709 13:04:51.338734 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.338906 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":284673,"threads_started":1,"wal-append.queue_time_us":484748}
I20260709 13:04:51.432627 12612 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:51.525813  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:51.527472  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:51.462764 12605 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:51.442029 12574 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.260179 12593 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.449167 12618 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.394328 12610 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.315650 12152 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:51.454607 12366 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:51.671118  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 360900us
I20260709 13:04:51.919137  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:51.921615  9861 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.007s	sys 0.011s
I20260709 13:04:51.673900  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:51.922906  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:51.923897  9857 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:51.575924  9019 catalog_manager.cc:5697] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
W20260709 13:04:51.596434  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.924820  9270 consensus_meta.cc:287] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.001s
I20260709 13:04:51.686005  9855 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:51.925804  9270 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
W20260709 13:04:51.678313  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:51.922287  9861 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.008s	sys 0.011s
I20260709 13:04:51.756868  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2139) took 1907 ms. Trace:
W20260709 13:04:51.930401  9825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.020s	sys 0.003s
W20260709 13:04:51.930954  9825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.020s	sys 0.003s
W20260709 13:04:51.502032 12374 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.022s	sys 0.054s
I20260709 13:04:51.923295  9122 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
W20260709 13:04:51.933972  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 406646us
W20260709 13:04:51.934490  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 631932us
I20260709 13:04:51.935470  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:51.925940  9019 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 4, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 4 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.500335 12573 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:51.941604  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:51.942616  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:51.928344  9120 rpcz_store.cc:276] 0709 13:04:49.849447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.026769 (+177322us) service_pool.cc:224] Handling call
0709 13:04:50.027115 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:50.027470 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.027500 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:50.035550 (+  8050us) log.cc:844] Serialized 52 byte log entry
0709 13:04:50.405613 (+370063us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:50.424493 (+ 18880us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.453623 (+ 29130us) log.cc:844] Serialized 14 byte log entry
0709 13:04:50.453841 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.453949 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:50.597889 (+143940us) spinlock_profiling.cc:243] Waited 9.55 ms on lock 0x7b48000d0a58. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:04:51.733165 (+1135276us) raft_consensus.cc:1685] finished
0709 13:04:51.733205 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:51.756759 (+ 23554us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28615,"prepare.queue_time_us":19004,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":36493,"raft.queue_time_us":191351,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":183953,"spinlock_wait_cycles":144301184,"thread_start_us":245371,"threads_started":1,"wal-append.queue_time_us":619280}
W20260709 13:04:51.852563 12458 consensus_meta.cc:287] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.421s	user 0.026s	sys 0.000s
W20260709 13:04:51.850615 12558 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.488s	user 0.011s	sys 0.001s
W20260709 13:04:51.890766 12642 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:04:51.726075 12474 consensus_meta.cc:287] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.250s	user 0.013s	sys 0.004s
I20260709 13:04:51.797360 12371 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:51.855659 11805 consensus_meta.cc:287] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.219s	user 0.005s	sys 0.005s
W20260709 13:04:51.874018 12584 consensus_meta.cc:287] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.428s	user 0.007s	sys 0.004s
I20260709 13:04:51.966485 12573 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:51.901662 12500 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.663s	user 0.012s	sys 0.002s
I20260709 13:04:51.973904  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:51.974417  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:51.544438 12566 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:51.987190 12640 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:51.847909 12291 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:51.893260 12356 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:51.932009 12374 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.068s	user 0.038s	sys 0.068s
I20260709 13:04:51.851257 12494 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:51.571679 12543 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:04:51.577064 12613 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:04:51.915659 12069 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:51.989132 12474 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:51.797058 12269 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:51.572013 12639 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:04:51.733438 12493 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:51.952904 12626 consensus_meta.cc:287] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.002s
I20260709 13:04:51.921425 12612 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:51.955941  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:51.908356 12559 consensus_meta.cc:287] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.370s	user 0.005s	sys 0.013s
I20260709 13:04:51.577215 12649 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:51.991612 12426 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.001s
I20260709 13:04:51.989918 12646 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:51.997015  9122 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:51.997241  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:51.644618 12634 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:04:51.583630 12617 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.674s
I20260709 13:04:52.003461  9862 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:52.000022 12566 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:52.005100  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:51.986532 12558 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:51.990680 12565 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:52.268935  9190 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.023s	sys 0.019s
W20260709 13:04:52.392762  9190 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.024s	sys 0.020s
I20260709 13:04:52.388764  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 572) took 1088 ms. Trace:
I20260709 13:04:52.394027  9122 rpcz_store.cc:276] 0709 13:04:51.300735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.970897 (+670162us) service_pool.cc:224] Handling call
0709 13:04:52.388693 (+417796us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:52.229697  9236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.004s	sys 0.038s
W20260709 13:04:52.395639  9236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.004s	sys 0.044s
I20260709 13:04:52.384954  9855 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:04:52.410060  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 576) took 1108 ms. Trace:
I20260709 13:04:52.410696  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 926) took 1219 ms. Trace:
I20260709 13:04:51.993234 12633 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:52.323827 12583 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:52.005415 12662 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.002516 12152 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:52.410619  9120 rpcz_store.cc:276] 0709 13:04:51.301776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.004827 (+703051us) service_pool.cc:224] Handling call
0709 13:04:52.409982 (+405155us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:52.384878  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 449449us
I20260709 13:04:52.425040  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:52.425936  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:52.466081  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 470084us
W20260709 13:04:52.050999 12659 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:04:52.411396  9855 rpcz_store.cc:276] 0709 13:04:51.191005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.940762 (+749757us) service_pool.cc:224] Handling call
0709 13:04:52.410625 (+469863us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:52.325062 12644 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:04:52.056743 12609 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:52.491668  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:52.054369  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 981) took 1268 ms. Trace:
I20260709 13:04:52.521766 12662 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:52.522560 12656 log.cc:927] Time spent T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.345s	user 0.000s	sys 0.000s
I20260709 13:04:52.064196 12458 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:52.054085 12593 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:52.062949 12618 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:52.522817 12633 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:52.717022  9235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.006s	sys 0.020s
W20260709 13:04:52.728330  9235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.007s	sys 0.021s
I20260709 13:04:52.059933 12660 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.007289 12500 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:52.685708  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:52.051301 12422 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:52.394274 12545 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:52.679382  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 426256us
I20260709 13:04:52.581935  9118 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:52.732227  9122 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:52.730943  9860 rpcz_store.cc:276] 0709 13:04:50.786279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.787178 (+   899us) service_pool.cc:224] Handling call
0709 13:04:50.787419 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.309154 (+521735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.309183 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.309224 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:51.309252 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.054116 (+744864us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b540070f154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:52.054168 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:52.054311 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199453952}
I20260709 13:04:52.064105 12574 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:52.389251 12559 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:52.491153 12494 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:52.578243 12649 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.302s
I20260709 13:04:52.386754 12646 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:52.388309 12426 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:52.757467  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:52.757011  9192 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:04:52.531890  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.777251  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:52.774117  9825 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.683s (SENT)
I20260709 13:04:52.523250 12069 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:52.717718 12366 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.008s	sys 0.017s
I20260709 13:04:52.772866 12422 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:52.774638 12545 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:52.391065 12566 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:52.515081 12565 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:52.496565 12568 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:52.616343 12573 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:52.985322 12572 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.186744 12574 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:54.188520 12568 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:52.523017 12583 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.190531 12568 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:52.004913 12606 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:52.004009 12640 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:52.581249 12653 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.681438 12657 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:52.773574 12660 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:54.422107  9194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2013) took 3462 ms (client timeout 2107 ms). Trace:
I20260709 13:04:54.422981 12660 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:54.422663  9194 rpcz_store.cc:269] 0709 13:04:50.959677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.096685 (+137008us) service_pool.cc:224] Handling call
0709 13:04:54.421589 (+3324904us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b54000eef54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:54.421874 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77468928}
W20260709 13:04:54.017498  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.279s	user 0.000s	sys 0.014s
W20260709 13:04:54.017802  9163 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.604s	user 0.003s	sys 0.023s
W20260709 13:04:54.020047  9087 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 2.088s	user 0.000s	sys 0.021s
I20260709 13:04:53.098845 12474 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.187215 12572 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:53.152240  9268 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.222s	user 0.011s	sys 0.019s
I20260709 13:04:54.038317  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2012) took 2741 ms. Trace:
I20260709 13:04:52.389880 12617 consensus_queue.cc:1243] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:04:54.272993  9265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1820) took 3562 ms (client timeout 2179 ms). Trace:
I20260709 13:04:54.021973  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 578) took 2719 ms. Trace:
I20260709 13:04:54.027701  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 577) took 2725 ms. Trace:
I20260709 13:04:52.390372 12626 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:52.878695 12610 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.435956  9120 rpcz_store.cc:276] 0709 13:04:51.302021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.796462 (+1494441us) service_pool.cc:224] Handling call
0709 13:04:52.796796 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.797003 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.797032 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:52.797072 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:52.797101 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.797181 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.027403 (+1230222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:52.957034 12647 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.253s	user 0.000s	sys 0.000s
W20260709 13:04:53.014719 12627 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 1.548s	user 0.001s	sys 0.008s
I20260709 13:04:52.991420 12575 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:52.767551 12458 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:52.797829 12639 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.384420  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2015) took 1997 ms. Trace:
W20260709 13:04:52.851408 12567 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.933s	user 0.016s	sys 0.006s
W20260709 13:04:53.078397 12510 consensus_meta.cc:287] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.588s	user 0.008s	sys 0.003s
W20260709 13:04:53.045446  9827 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.110s	user 0.015s	sys 0.061s
I20260709 13:04:54.385836 12494 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:53.151835  9238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.223s	user 0.025s	sys 0.049s
W20260709 13:04:52.578038 12634 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 13:04:52.780655  9826 connection.cc:256] RPC call timeout handler was delayed by -1.0466196160000436s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 13:04:53.152016 12374 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.012s	sys 0.018s
W20260709 13:04:53.075668 12678 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:04:54.446929 12572 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:54.435411  9265 rpcz_store.cc:269] 0709 13:04:50.707412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.708983 (+  1571us) service_pool.cc:224] Handling call
0709 13:04:54.269332 (+3560349us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54000f9bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:54.269548 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626957184}
I20260709 13:04:54.448946 12426 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.449337  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:54.450938  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 979) took 3665 ms. Trace:
W20260709 13:04:53.140247 12630 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.695s	user 0.008s	sys 0.015s
I20260709 13:04:54.451436  9854 rpcz_store.cc:276] 0709 13:04:50.785702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.786532 (+   830us) service_pool.cc:224] Handling call
0709 13:04:50.806690 (+ 20158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.276495 (+469805us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.276538 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.276592 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:51.276634 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.450365 (+3173731us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b540070e4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.450438 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.450672 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079894912}
I20260709 13:04:54.452631  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2199) took 3846 ms. Trace:
W20260709 13:04:53.139182  9269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1812) took 2589 ms (client timeout 1999 ms). Trace:
W20260709 13:04:54.453128  9827 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.515s	user 0.017s	sys 0.076s
I20260709 13:04:54.454190 12639 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:54.455407 12374 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.868s	user 0.017s	sys 0.029s
I20260709 13:04:54.455559  9120 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:04:54.456485  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 579) took 3153 ms. Trace:
I20260709 13:04:54.436498 12640 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:54.457059 12639 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:53.828485 12612 consensus_meta.cc:287] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.419s	user 0.011s	sys 0.000s
W20260709 13:04:53.112664 12277 consensus_meta.cc:287] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.852s	user 0.017s	sys 0.001s
W20260709 13:04:53.045446 12498 consensus_meta.cc:287] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.666s	user 0.007s	sys 0.006s
I20260709 13:04:54.478328 12498 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:54.480767 12622 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 3.033s	user 0.001s	sys 0.040s
I20260709 13:04:54.481209 12498 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.026973  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:54.482726  9198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2019) took 1685 ms (client timeout 1725 ms). Trace:
W20260709 13:04:54.483117  9198 rpcz_store.cc:269] 0709 13:04:52.797819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.799465 (+  1646us) service_pool.cc:224] Handling call
0709 13:04:54.482621 (+1683156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.484709  9266 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:04:53.079278 12376 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.536s	user 0.007s	sys 0.001s
I20260709 13:04:54.486619  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2143) took 3732 ms. Trace:
I20260709 13:04:54.486091 12376 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:53.126690  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1806) took 2578 ms. Trace:
I20260709 13:04:52.409650 12269 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:54.488545 12639 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.489012 12269 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.489984  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2193) took 4399 ms. Trace:
I20260709 13:04:54.489665 12639 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.490232 12277 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:54.490414  9118 rpcz_store.cc:276] 0709 13:04:50.090937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.209195 (+118258us) service_pool.cc:224] Handling call
0709 13:04:50.235543 (+ 26348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:52.774087 (+2538544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.489266 (+1715179us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b5400267354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.489730 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5395920768}
I20260709 13:04:54.164191 12558 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.385905 12583 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:53.046630 12497 raft_consensus.cc:3055] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:54.492429 12639 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.492839 12497 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:54.493000 12277 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.494490 12692 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.494431 12497 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:52.816669  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:54.495218 12692 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:54.496220  9855 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
W20260709 13:04:54.496914  9855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 925) took 3307 ms (client timeout 3260 ms). Trace:
W20260709 13:04:54.497602  9855 rpcz_store.cc:269] 0709 13:04:51.189881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.783522 (+1593641us) service_pool.cc:224] Handling call
0709 13:04:54.496791 (+1713269us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:54.498611  9862 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 971) took 4036 ms (client timeout 3380 ms). Trace:
I20260709 13:04:54.498728 12626 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.487093  9117 rpcz_store.cc:276] 0709 13:04:50.754257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.764968 (+ 10711us) service_pool.cc:224] Handling call
0709 13:04:50.765345 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.151428 (+386083us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:04:51.391473 (+240045us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.681912 (+290439us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.681973 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:51.682033 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.486061 (+2804028us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400265f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.486134 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.486414 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138124,"prepare.queue_time_us":288815,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":86,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":887413504,"thread_start_us":498329,"threads_started":2,"wal-append.queue_time_us":596044,"wal-append.run_cpu_time_us":11490,"wal-append.run_wall_time_us":1850535}
I20260709 13:04:52.781833  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:52.990351 12675 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:53.137725 12603 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.410738  9114 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:04:54.412308 12559 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.412338  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2014) took 2026 ms. Trace:
I20260709 13:04:53.122051 12633 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.501756  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 974) took 4013 ms. Trace:
W20260709 13:04:54.413748  9270 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2315) took 1686 ms (client timeout 1598 ms). Trace:
W20260709 13:04:54.413914  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 1.63964e+06us
I20260709 13:04:53.071902 12654 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.500645 12675 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:52.798645 12593 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.107293  9272 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
W20260709 13:04:54.502411  9270 rpcz_store.cc:269] 0709 13:04:52.727375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.776867 (+ 49492us) service_pool.cc:224] Handling call
0709 13:04:54.413490 (+1636623us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:54.419584 12366 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.220s	user 0.010s	sys 0.028s
W20260709 13:04:54.423715 12632 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 2.934s	user 0.011s	sys 0.029s
W20260709 13:04:54.018108  9164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.067s	user 0.017s	sys 0.026s
I20260709 13:04:54.501873  9191 rpcz_store.cc:276] 0709 13:04:52.386332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.386464 (+   132us) service_pool.cc:224] Handling call
0709 13:04:52.386818 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:52.387059 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:52.387082 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:52.387520 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:04:52.845531 (+458011us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:52.845570 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.845649 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.411800 (+1566151us) raft_consensus.cc:1685] finished
0709 13:04:54.411849 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.412116 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":457707,"threads_started":1,"wal-append.queue_time_us":2021336}
W20260709 13:04:54.427062  9163 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.013s	user 0.011s	sys 0.030s
W20260709 13:04:54.427428  9087 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 2.496s	user 0.002s	sys 0.030s
I20260709 13:04:54.187548 12422 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.510627 12593 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.511602 12422 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:54.513470 12593 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:52.955864 12673 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:54.488488  9271 rpcz_store.cc:276] 0709 13:04:50.548120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.548248 (+   128us) service_pool.cc:224] Handling call
0709 13:04:50.646606 (+ 98358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.772492 (+125886us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:50.802394 (+ 29902us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.052598 (+1250204us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b18003df0c0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:04:52.229388 (+176790us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:52.230104 (+   716us) log.cc:844] Serialized 26 byte log entry
0709 13:04:52.230283 (+   179us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:52.230578 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.717441 (+486863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.717558 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:53.126391 (+408833us) raft_consensus.cc:1685] finished
0709 13:04:53.126421 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:53.126611 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":323890,"prepare.queue_time_us":30110,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":84,"raft.queue_time_us":1084342,"raft.run_cpu_time_us":470,"raft.run_wall_time_us":106501,"spinlock_wait_cycles":4959488,"thread_start_us":663000,"threads_started":2,"wal-append.queue_time_us":2165077,"wal-append.run_cpu_time_us":3574,"wal-append.run_wall_time_us":1219086}
I20260709 13:04:54.514513 12654 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.514820 12673 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.429555 12069 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.516966  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2313) took 1901 ms. Trace:
I20260709 13:04:54.517491  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:54.517395  9271 rpcz_store.cc:276] 0709 13:04:52.615973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.515674 (+1899701us) service_pool.cc:224] Handling call
0709 13:04:54.516208 (+   534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.516575 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.516603 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.516646 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:54.516682 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.516747 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.516885 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:52.401273 12543 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.435854  9122 rpcz_store.cc:276] 0709 13:04:51.302298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.834100 (+1531802us) service_pool.cc:224] Handling call
0709 13:04:54.020251 (+1186151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.020933 (+   682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.021011 (+    78us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.021120 (+   109us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:54.021197 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.021329 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.021631 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.442945 12668 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:04:54.441924  9268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.506s	user 0.016s	sys 0.029s
I20260709 13:04:54.440583  9192 rpcz_store.cc:276] 0709 13:04:52.386653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.491166 (+104513us) service_pool.cc:224] Handling call
0709 13:04:52.756610 (+265444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.231345 (+1474735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.384196 (+152851us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:54.446313  9238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.517s	user 0.030s	sys 0.056s
W20260709 13:04:53.074919 12655 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.462s	user 0.000s	sys 0.000s
W20260709 13:04:54.454882  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:52.866353 12664 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:54.523538 12269 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.533113  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2012) took 3573 ms. Trace:
I20260709 13:04:54.534490 12675 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:54.534873  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 2.60564e+06us
I20260709 13:04:54.533541  9193 rpcz_store.cc:276] 0709 13:04:50.959370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.959592 (+   222us) service_pool.cc:224] Handling call
0709 13:04:50.959939 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.960330 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.960389 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.077760 (+117371us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.231832 (+154072us) spinlock_profiling.cc:243] Waited 9.09 ms on lock 0x7b1800392800. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:04:51.496728 (+264896us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:51.680749 (+184021us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.681242 (+   493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.681336 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.532540 (+2851204us) raft_consensus.cc:1685] finished
0709 13:04:54.532682 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.532921 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":183518,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":127581,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":20004,"spinlock_wait_cycles":20899584,"thread_start_us":259536,"threads_started":1,"wal-append.queue_time_us":1411699}
I20260709 13:04:54.449158 12567 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.537022  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:54.449239 12494 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.538946 12567 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.540216  9853 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:04:54.540745 12494 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:54.541272 12494 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:54.541700 12494 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:54.502300  9859 rpcz_store.cc:276] 0709 13:04:50.488429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.533125 (+ 44696us) service_pool.cc:224] Handling call
0709 13:04:50.533493 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:50.568303 (+ 34810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.568352 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:50.571381 (+  3029us) log.cc:844] Serialized 52 byte log entry
0709 13:04:51.111319 (+539938us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:51.111377 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.111469 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:52.994041 (+1882572us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:54.501161 (+1507120us) raft_consensus.cc:1685] finished
0709 13:04:54.501216 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.501599 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21530,"raft.queue_time_us":21807,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"spinlock_wait_cycles":37836672,"thread_start_us":539397,"threads_started":1,"wal-append.queue_time_us":874395}
W20260709 13:04:54.457870  9269 rpcz_store.cc:269] 0709 13:04:50.549997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.550305 (+   308us) service_pool.cc:224] Handling call
0709 13:04:53.138866 (+2588561us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54003a25d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:53.139086 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1405069568}
I20260709 13:04:54.406538 12574 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:54.545879 12574 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.543730  9859 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:54.459882  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 988) took 3410 ms. Trace:
I20260709 13:04:54.547147  9854 rpcz_store.cc:276] 0709 13:04:51.049478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.453358 (+3403880us) service_pool.cc:224] Handling call
0709 13:04:54.454133 (+   775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.459382 (+  5249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.459409 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.459474 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:54.459547 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.459612 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.459776 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.548393  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:54.460659  9265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2311) took 1867 ms (client timeout 1969 ms). Trace:
W20260709 13:04:54.550464  9265 rpcz_store.cc:269] 0709 13:04:52.592837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.448858 (+1856021us) service_pool.cc:224] Handling call
0709 13:04:54.460556 (+ 11698us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.460219 12510 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:54.460691 12646 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.454587 12458 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:54.461290  9826 connection.cc:256] RPC call timeout handler was delayed by -1.0421196160000363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
I20260709 13:04:54.431208  9190 rpcz_store.cc:276] 0709 13:04:51.296611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.296763 (+   152us) service_pool.cc:224] Handling call
0709 13:04:51.297176 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:51.315250 (+ 18074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.315277 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:51.315735 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:04:52.393288 (+1077553us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:52.393336 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.393429 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.037563 (+1644134us) raft_consensus.cc:1685] finished
0709 13:04:54.037610 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.038028 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17557,"raft.queue_time_us":17754,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":39465,"thread_start_us":980948,"threads_started":1,"wal-append.queue_time_us":1571592,"wal-append.run_cpu_time_us":2710,"wal-append.run_wall_time_us":1418033}
I20260709 13:04:54.462677 12575 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:54.456818  9120 rpcz_store.cc:276] 0709 13:04:51.302583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.454514 (+3151931us) service_pool.cc:224] Handling call
0709 13:04:54.455071 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.456193 (+  1122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.456417 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.473254 12612 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:54.476213 12614 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 3.242s	user 0.007s	sys 0.020s
I20260709 13:04:54.477087  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.488641 12355 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:04:54.499102  9862 rpcz_store.cc:269] 0709 13:04:50.462469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.464390 (+  1921us) service_pool.cc:224] Handling call
0709 13:04:54.498159 (+4033769us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400524254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:54.498387 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3536080256}
W20260709 13:04:54.500631  9189 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2018) took 2075 ms (client timeout 2171 ms). Trace:
I20260709 13:04:53.123066 12662 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.415197 12653 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.506220  9272 raft_consensus.cc:606] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:54.598685  9272 raft_consensus.cc:614] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:04:54.426730  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.688s	user 0.000s	sys 0.025s
I20260709 13:04:54.415408 12657 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:54.520494 12673 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:54.520968  9164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.570s	user 0.020s	sys 0.033s
W20260709 13:04:54.605904  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.521622 12069 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:54.521608  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 2.59031e+06us
I20260709 13:04:54.521522 12566 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.522101  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 973) took 4058 ms. Trace:
I20260709 13:04:54.529495  9855 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:54.529788 12668 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:54.531313 12366 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:54.549960  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:54.546712  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 927) took 3355 ms. Trace:
I20260709 13:04:54.632505  9859 rpcz_store.cc:276] 0709 13:04:51.191298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.542728 (+3351430us) service_pool.cc:224] Handling call
0709 13:04:54.543164 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.546453 (+  3289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.546640 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.527566 12593 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.583313  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2316) took 1851 ms. Trace:
I20260709 13:04:54.583612 12510 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:54.549412 12664 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:54.591164  9189 rpcz_store.cc:269] 0709 13:04:52.425184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.781445 (+356261us) service_pool.cc:224] Handling call
0709 13:04:54.500376 (+1718931us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.601217 12654 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:54.628851  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 230053us
W20260709 13:04:54.635948  9194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2020) took 1837 ms (client timeout 2082 ms). Trace:
I20260709 13:04:54.640200  9271 rpcz_store.cc:276] 0709 13:04:52.731510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.580560 (+1849050us) service_pool.cc:224] Handling call
0709 13:04:54.582414 (+  1854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.582906 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.582937 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.582997 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:54.583038 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.583094 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.583235 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.638679  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 989) took 3575 ms. Trace:
I20260709 13:04:54.590787 12355 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:54.626729  9861 rpcz_store.cc:276] 0709 13:04:50.463099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.495766 (+ 32667us) service_pool.cc:224] Handling call
0709 13:04:50.644423 (+148657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.644845 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.644880 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.665035 (+ 20155us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.160875 (+495840us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:51.935897 (+775022us) log.cc:844] Serialized 14 byte log entry
0709 13:04:52.200215 (+264318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:52.200304 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.422453 (+2222149us) spinlock_profiling.cc:243] Waited 7.56 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:54.521484 (+ 99031us) raft_consensus.cc:1685] finished
0709 13:04:54.521616 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.521919 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":291396,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":66782,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":13848,"spinlock_wait_cycles":17381760,"thread_start_us":1406416,"threads_started":3,"wal-append.queue_time_us":697842}
I20260709 13:04:54.750914  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:54.708375  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:54.755548  9859 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:54.721266  9854 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:54.453125  9123 rpcz_store.cc:276] 0709 13:04:50.605764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.608725 (+  2961us) service_pool.cc:224] Handling call
0709 13:04:50.609053 (+   328us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:50.647644 (+ 38591us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:04:50.685018 (+ 37374us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.204280 (+519262us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:51.401439 (+197159us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b2c00000f48. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:51.426571 (+ 25132us) log.cc:844] Serialized 52 byte log entry
0709 13:04:51.426781 (+   210us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:04:51.426838 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.426918 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.439095 (+3012177us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b1800344680. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:54.452035 (+ 12940us) raft_consensus.cc:1685] finished
0709 13:04:54.452059 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.452395 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36736,"prepare.queue_time_us":41854,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":7892,"raft.queue_time_us":180816,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":30851,"spinlock_wait_cycles":51361920,"thread_start_us":423480,"threads_started":1,"wal-append.queue_time_us":539895}
I20260709 13:04:54.753633  9855 rpcz_store.cc:276] 0709 13:04:51.053310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.499098 (+3445788us) service_pool.cc:224] Handling call
0709 13:04:54.499575 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.628188 (+128613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.628469 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:54.753620  9194 rpcz_store.cc:269] 0709 13:04:52.798214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.476655 (+1678441us) service_pool.cc:224] Handling call
0709 13:04:54.635853 (+159198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.706420  9857 raft_consensus.cc:2393] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:54.591527 12567 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.724217  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:54.707489  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:54.654014  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:56116 (request call id 25) took 3170 ms. Trace:
W20260709 13:04:54.757375  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 220551us
I20260709 13:04:54.721505  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2135) took 4873 ms. Trace:
I20260709 13:04:54.708873  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:54.719882  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 294288us
I20260709 13:04:54.769742  9272 rpcz_store.cc:276] 0709 13:04:51.428555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.106759 (+2678204us) service_pool.cc:224] Handling call
0709 13:04:54.599157 (+492398us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.773556  9117 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:54.684846 12674 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.721976 12657 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.784022 12113 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.784868 12113 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:54.785285 12625 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 3.338s	user 0.003s	sys 0.032s
I20260709 13:04:54.787318  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 970) took 4327 ms. Trace:
I20260709 13:04:54.752061 12653 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:54.653437 12593 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.789660  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2317) took 2056 ms. Trace:
I20260709 13:04:54.787685  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 916) took 4373 ms. Trace:
I20260709 13:04:54.795961  9271 rpcz_store.cc:276] 0709 13:04:52.732988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.776847 (+2043859us) service_pool.cc:224] Handling call
0709 13:04:54.788753 (+ 11906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.789289 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.789316 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.789378 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:54.789409 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.789471 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.789585 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:04:54.680863 12629 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 3.206s	user 0.029s	sys 0.083s
I20260709 13:04:54.753436 12566 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.706523 12612 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.683933 12583 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.773030 12494 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.802448  9118 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:54.803158  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:54.806308 12423 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:54.807214 12423 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.809898 12494 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.818605  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:54.818177  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:54.820480  9120 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:54.811395 12523 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:54.770246  9115 rpcz_store.cc:276] 0709 13:04:49.848043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:49.943464 (+ 95421us) service_pool.cc:224] Handling call
0709 13:04:49.946723 (+  3259us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:50.214288 (+267565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.214324 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:50.271719 (+ 57395us) log.cc:844] Serialized 582 byte log entry
0709 13:04:51.075729 (+804010us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:51.075889 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.075957 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.475571 (+3399614us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b2400168d88. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765347d9c4 00007f765347bacb 00007f7653636dc9 00007f7653636a55 00007f765363609d 00007f765626810e 00007f765627f88c 00007f765627ebb6 00007f765627e7d8 00007f7656288221
0709 13:04:54.720871 (+245300us) raft_consensus.cc:1685] finished
0709 13:04:54.720897 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.721309 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12474,"raft.queue_time_us":30885,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":4638,"spinlock_wait_cycles":12575488,"thread_start_us":1041046,"threads_started":2,"wal-append.queue_time_us":887742}
I20260709 13:04:54.825361  9862 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:04:54.820323  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 109526us
I20260709 13:04:54.790977 12355 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.787757  9856 rpcz_store.cc:276] 0709 13:04:50.459622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.495842 (+ 36220us) service_pool.cc:224] Handling call
0709 13:04:50.496368 (+   526us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:04:51.120004 (+623636us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:04:51.273452 (+153448us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.573949 (+300497us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:51.574651 (+   702us) log.cc:844] Serialized 52 byte log entry
0709 13:04:51.574804 (+   153us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:51.574848 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.574936 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.534625 (+2959689us) raft_consensus.cc:1685] finished
0709 13:04:54.534671 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.787080 (+252409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6075,"prepare.queue_time_us":332027,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":441036,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":2864,"thread_start_us":910396,"threads_started":3,"wal-append.queue_time_us":768834}
I20260709 13:04:54.829584  9855 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:54.823181 12113 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.834494  9271 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:04:54.832418  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2312) took 2214 ms. Trace:
I20260709 13:04:54.836500  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2318) took 2102 ms. Trace:
I20260709 13:04:54.836956  9271 rpcz_store.cc:276] 0709 13:04:52.734176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.803794 (+2069618us) service_pool.cc:224] Handling call
0709 13:04:54.833972 (+ 30178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.834915 (+   943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.836443 (+  1528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.796209  9858 rpcz_store.cc:276] 0709 13:04:50.414659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.415807 (+  1148us) service_pool.cc:224] Handling call
0709 13:04:50.416080 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.455539 (+ 39459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.455567 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.467178 (+ 11611us) log.cc:844] Serialized 26 byte log entry
0709 13:04:50.871058 (+403880us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:50.871112 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:50.871204 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.784295 (+3913091us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:54.787176 (+  2881us) raft_consensus.cc:1685] finished
0709 13:04:54.787202 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.787509 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67194,"raft.queue_time_us":39727,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":38748,"spinlock_wait_cycles":36732160,"thread_start_us":403476,"threads_started":1,"wal-append.queue_time_us":979216}
I20260709 13:04:54.837884 12511 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:54.840690 12511 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:54.841296 12552 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.842298 12355 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:54.843119 12511 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.847131 12664 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.848069 12376 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:54.850380  9162 connection.cc:256] RPC call timeout handler was delayed by -1.4942717010000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19690000000000002s for a response
W20260709 13:04:54.852037  9266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1833) took 3771 ms (client timeout 1609 ms). Trace:
I20260709 13:04:54.825232 12523 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:54.852496  9266 rpcz_store.cc:269] 0709 13:04:51.080392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.143524 (+ 63132us) service_pool.cc:224] Handling call
0709 13:04:54.851788 (+3708264us) spinlock_profiling.cc:243] Waited 3.29 s on lock 0x7b54000fb9d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:54.851971 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7560189696}
I20260709 13:04:54.831840 12668 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:54.854249 12523 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.830886 12674 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:54.835731  9270 rpcz_store.cc:276] 0709 13:04:52.615632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.509094 (+1893462us) service_pool.cc:224] Handling call
0709 13:04:54.510890 (+  1796us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:54.511643 (+   753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.511671 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:54.512618 (+   947us) log.cc:844] Serialized 52 byte log entry
0709 13:04:54.679498 (+166880us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:54.828878 (+149380us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.829129 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.829203 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.829239 (+    36us) raft_consensus.cc:1685] finished
0709 13:04:54.829250 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.829422 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68307,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":45,"raft.queue_time_us":13877,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":46175,"thread_start_us":219212,"threads_started":2,"wal-append.queue_time_us":233361}
I20260709 13:04:54.854156  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:54.855897  9271 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:54.856886  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2320) took 2120 ms. Trace:
I20260709 13:04:54.832224 12653 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:54.857232  9271 rpcz_store.cc:276] 0709 13:04:52.736930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.855001 (+2118071us) service_pool.cc:224] Handling call
0709 13:04:54.855445 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.856644 (+  1199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.856836 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.860152  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2005) took 4440 ms. Trace:
I20260709 13:04:54.862252  9118 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:54.861310  9197 rpcz_store.cc:276] 0709 13:04:50.419763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.420111 (+   348us) service_pool.cc:224] Handling call
0709 13:04:50.495941 (+ 75830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:51.076092 (+580151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.859615 (+3783523us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54000a7654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.859972 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724762496}
I20260709 13:04:54.824662 12567 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.863816  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2141) took 4668 ms. Trace:
I20260709 13:04:54.869477  9115 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:04:54.870906  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2007) took 4450 ms. Trace:
W20260709 13:04:54.871111  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 735) took 3696 ms (client timeout 2590 ms). Trace:
I20260709 13:04:54.866194  9116 rpcz_store.cc:276] 0709 13:04:50.195243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.231229 (+ 35986us) service_pool.cc:224] Handling call
0709 13:04:50.263567 (+ 32338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:50.792393 (+528826us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:50.792428 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:50.792842 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 13:04:51.203110 (+410268us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:51.203152 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:51.203231 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:51.274925 (+ 71694us) raft_consensus.cc:1685] finished
0709 13:04:51.274945 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.863379 (+3588434us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b54001e3f50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:54.863626 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":54383,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"spinlock_wait_cycles":73402624,"thread_start_us":938230,"threads_started":2,"wal-append.queue_time_us":475422,"wal-append.run_cpu_time_us":2101,"wal-append.run_wall_time_us":1055328}
I20260709 13:04:54.873400  9117 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:54.879869  9123 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
W20260709 13:04:54.880100  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 102623us
W20260709 13:04:54.871573  9267 rpcz_store.cc:269] 0709 13:04:51.172234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.867475 (+3695241us) service_pool.cc:211] Skipping call since client already timed out
0709 13:04:54.867668 (+   193us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:04:54.872929  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:54.882812  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2202) took 4276 ms. Trace:
I20260709 13:04:54.883306 12550 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:04:54.867897  9826 leader_election.cc:336] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.519s (SENT)
I20260709 13:04:54.884610  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:54.885155  9197 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:04:54.886035  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47444us
I20260709 13:04:54.871472  9195 rpcz_store.cc:276] 0709 13:04:50.420326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.421058 (+   732us) service_pool.cc:224] Handling call
0709 13:04:50.448977 (+ 27919us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:51.075449 (+626472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.866105 (+3790656us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400217354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.870727 (+  4622us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838443776}
I20260709 13:04:54.874310 12423 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.888017  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:54.888471  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2310) took 2296 ms. Trace:
I20260709 13:04:54.889389 12618 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:54.889974  9263 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1824) took 3994 ms (client timeout 2150 ms). Trace:
W20260709 13:04:54.890378  9263 rpcz_store.cc:269] 0709 13:04:50.895802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.895956 (+   154us) service_pool.cc:224] Handling call
0709 13:04:54.889624 (+3993668us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b54000ebad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:54.889787 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109487360}
I20260709 13:04:54.891633 12628 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:04:54.888899  9264 rpcz_store.cc:276] 0709 13:04:52.591798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.591939 (+   141us) service_pool.cc:224] Handling call
0709 13:04:52.592338 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.445970 (+1853632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:54.446024 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.446093 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:04:54.446139 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.888083 (+441944us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54003fcad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.888127 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.888291 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4226664832}
I20260709 13:04:54.894091  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.894307  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.894457  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.894902  9195 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:54.898377  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 585) took 4355 ms. Trace:
I20260709 13:04:54.898816  9196 rpcz_store.cc:276] 0709 13:04:50.542782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.545033 (+  2251us) service_pool.cc:224] Handling call
0709 13:04:50.545402 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:51.085434 (+540032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.419722 (+3334288us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b540001e620. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f765665bdc0 00007f76566e7a02 00007f76566e3dd6 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:54.897913 (+478191us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54004169d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.898206 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713511424}
I20260709 13:04:54.871555 12545 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.902153  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.904526  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2201) took 4298 ms. Trace:
I20260709 13:04:54.905020  9114 rpcz_store.cc:276] 0709 13:04:50.606245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.608975 (+  2730us) service_pool.cc:224] Handling call
0709 13:04:50.609207 (+   232us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.501382 (+3892175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.904259 (+402877us) spinlock_profiling.cc:243] Waited 3.8 s on lock 0x7b54001202d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.904466 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8741556480}
I20260709 13:04:54.857546 12640 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.907550  9119 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 4: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:54.909929  9854 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:04:54.883257  9121 rpcz_store.cc:276] 0709 13:04:50.606476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.610246 (+  3770us) service_pool.cc:224] Handling call
0709 13:04:50.610496 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:50.765862 (+155366us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:51.161214 (+395352us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.929589 (+768375us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.941501 (+ 11912us) log.cc:844] Serialized 14 byte log entry
0709 13:04:51.951646 (+ 10145us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:51.951700 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:51.951828 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:53.145157 (+1193329us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b1800445c60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7557 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:54.882394 (+1737237us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400266954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.882468 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.882637 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77180,"prepare.queue_time_us":2012236,"prepare.run_cpu_time_us":144,"prepare.run_wall_time_us":9436,"raft.queue_time_us":101785,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":27196,"spinlock_wait_cycles":361043072,"thread_start_us":887973,"threads_started":3,"wal-append.queue_time_us":632110,"wal-append.run_cpu_time_us":23314,"wal-append.run_wall_time_us":2004845}
I20260709 13:04:54.912189  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.913476  9189 raft_consensus.cc:2410] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:04:54.917057  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:54.907274  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2301) took 4351 ms. Trace:
I20260709 13:04:54.919613  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:54.920481  9856 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:54.921073 12497 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:54.890949  9826 leader_election.cc:336] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.524s (SENT)
I20260709 13:04:54.909341  9120 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:54.927419  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:54.929718 12636 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.473s	user 0.029s	sys 0.140s
W20260709 13:04:54.930274 12636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.490s	user 0.029s	sys 0.141s
I20260709 13:04:54.928789  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 928) took 3727 ms. Trace:
I20260709 13:04:54.845129 12583 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:54.932636  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:54.872701 12565 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.932559  9861 rpcz_store.cc:276] 0709 13:04:51.197654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.775411 (+3577757us) service_pool.cc:224] Handling call
0709 13:04:54.801848 (+ 26437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.811358 (+  9510us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:54.811748 (+   390us) log.cc:844] Serialized 14 byte log entry
0709 13:04:54.924497 (+112749us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.924571 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:54.924627 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.924717 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.924880 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1479,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":1637,"raft.queue_time_us":17820,"thread_start_us":96247,"threads_started":1}
I20260709 13:04:54.943069  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.941860  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.943563  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2319) took 2205 ms. Trace:
I20260709 13:04:54.945425  9272 rpcz_store.cc:276] 0709 13:04:52.735448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.823956 (+2088508us) service_pool.cc:224] Handling call
0709 13:04:54.867146 (+ 43190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.936231 (+ 69085us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:54.936275 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:54.940327 (+  4052us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:54.940376 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.940443 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.940550 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:04:54.948706 12474 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:04:54.951946  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 97978.5us
I20260709 13:04:54.953037  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:54.953580 12692 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.954228 12291 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:54.954839  9238 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 685fa3ec557f4f3db02ec6ccce1d6f3e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:54.919951  9268 rpcz_store.cc:276] 0709 13:04:50.555821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.573456 (+ 17635us) service_pool.cc:224] Handling call
0709 13:04:50.573758 (+   302us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:51.181825 (+608067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:51.181865 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:51.218280 (+ 36415us) log.cc:844] Serialized 52 byte log entry
0709 13:04:51.434201 (+215921us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:04:51.540629 (+106428us) log.cc:844] Serialized 14 byte log entry
0709 13:04:53.146451 (+1605822us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b180044aa60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:54.495766 (+1349315us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b180044aa60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:54.538107 (+ 42341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.854849 (+316742us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b180044aa60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535c52a6 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1
0709 13:04:54.855350 (+   501us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b180044aa60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:04:54.906840 (+ 51490us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54004452d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.906886 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:54.906937 (+    51us) raft_consensus.cc:1685] finished
0709 13:04:54.906948 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:54.907121 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":98669,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":399108,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":17899,"spinlock_wait_cycles":1488531072,"thread_start_us":2798857,"threads_started":2,"wal-append.queue_time_us":508980,"wal-append.run_cpu_time_us":2752,"wal-append.run_wall_time_us":1251334}
I20260709 13:04:54.955698 12640 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.968317 12565 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:54.969148 12386 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:04:54.973666  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 976) took 4437 ms. Trace:
I20260709 13:04:54.974152  9853 rpcz_store.cc:276] 0709 13:04:50.536219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:50.536756 (+   537us) service_pool.cc:224] Handling call
0709 13:04:50.537154 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:54.540718 (+4003564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:54.973365 (+432647us) spinlock_profiling.cc:243] Waited 4 s on lock 0x7b54006216d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:54.973596 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9205429632}
W20260709 13:04:54.981552  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:54.983283  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.985929  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:54.986552  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:54.987357  9194 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:54.991376  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:54.934278 12543 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:54.956291  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.996613  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:54.996804  9196 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:54.997216  9191 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:54.998027  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:54.999020  9193 raft_consensus.cc:2368] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:04:54.959710  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:54.957152  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:54.960217  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:54.943456 12631 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 3.506s	user 0.006s	sys 0.024s
I20260709 13:04:55.013152  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:54.960600  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:55.015100  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 124455us
I20260709 13:04:55.015226  9192 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:55.055027  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2314) took 2438 ms. Trace:
I20260709 13:04:55.107398  9268 raft_consensus.cc:2410] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:55.156479  9264 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:54.955879  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:54.964509 12593 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:54.964789 12567 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:54.967760 12674 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:55.179119  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.179867  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:55.180564  9272 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
W20260709 13:04:55.180882  9857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 923) took 4000 ms (client timeout 2229 ms). Trace:
W20260709 13:04:55.181095  9825 connection.cc:256] RPC call timeout handler was delayed by -2.4133372729999678s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2343s for a response
W20260709 13:04:55.181272  9857 rpcz_store.cc:269] 0709 13:04:51.187904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:51.941901 (+753997us) service_pool.cc:224] Handling call
0709 13:04:55.180633 (+3238732us) spinlock_profiling.cc:243] Waited 2.74 s on lock 0x7b5400738f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:55.180800 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6309341952}
I20260709 13:04:55.182680  9858 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
W20260709 13:04:55.143812 12631 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 3.830s	user 0.012s	sys 0.032s
I20260709 13:04:55.184657 12566 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.189131  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.189173  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:55.192224  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:55.192500  9114 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:04:55.192977  9268 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:55.190069  9862 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: [6.9-8.10]
I20260709 13:04:55.190858  9121 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:55.191023  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:55.195911  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:55.191452  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.196122  9266 raft_consensus.cc:2410] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:04:55.193005  9265 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
W20260709 13:04:55.154569  9826 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.662s (SENT) [suppressed 26 similar messages]
I20260709 13:04:55.191834  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:55.107796 12668 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:55.202948  9122 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
W20260709 13:04:55.208706  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54150.2us
W20260709 13:04:55.202521  9119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 573) took 3900 ms (client timeout 2343 ms). Trace:
I20260709 13:04:55.209995  9267 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:55.212725  9859 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:04:55.213138  9859 pending_rounds.cc:85] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 6
I20260709 13:04:55.213438  9859 pending_rounds.cc:107] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:04:55.213709  9859 raft_consensus.cc:2889] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:55.213872 12703 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
W20260709 13:04:55.214490  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 281402us
I20260709 13:04:55.215313  9087 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:55.216495 12493 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:55.216903  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:04:55.217312  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55057.5us
I20260709 13:04:55.221006  9191 raft_consensus.cc:2410] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:55.185221 12566 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.221815  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.184664 12583 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:55.228523  9164 connection.cc:256] RPC call timeout handler was delayed by -1.8947015080000256s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.338s for a response
I20260709 13:04:55.231976  9115 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:55.211338  9119 rpcz_store.cc:269] 0709 13:04:51.301051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.323805 (+1022754us) service_pool.cc:224] Handling call
0709 13:04:55.197070 (+2873265us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b540023c3d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:55.201211 (+  4141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219998592}
I20260709 13:04:55.233170  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.234400 12605 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:55.234861 12605 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:55.235234 12605 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:55.236626  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.237419  9861 raft_consensus.cc:2393] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:55.238211  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.239286  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.239921  9861 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 4.
I20260709 13:04:55.240139  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.240734  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.241308  9861 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:55.241343  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.242202  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:55.242573  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.242787  9861 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:55.243243  9857 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:55.244288  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.249257  9194 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:55.251971  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.252821  9858 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
W20260709 13:04:55.250744  9826 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.707s (SENT)
I20260709 13:04:55.259671  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.261097  9856 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:55.264597  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.267392  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:55.222306 12640 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:55.263456  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:55.272257  9120 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:04:55.279230  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 52491.5us
I20260709 13:04:55.281673  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.259296 12543 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:55.283798  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.286314  9853 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:04:55.286656  9853 pending_rounds.cc:85] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 8
W20260709 13:04:55.288853  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 333184us
W20260709 13:04:55.290410  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 73541.5us
I20260709 13:04:55.291884  9853 pending_rounds.cc:107] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:04:55.292348  9853 raft_consensus.cc:2889] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:55.285728 12567 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:55.300784  9857 raft_consensus.cc:2410] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:04:55.255990 12692 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:04:55.304636  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:55.305665  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.315328  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 83035us
W20260709 13:04:55.316368  9826 leader_election.cc:336] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.827s (SENT)
I20260709 13:04:55.316793  9826 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:55.318003 12566 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:55.318141  9826 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.904s (SENT)
I20260709 13:04:55.318697 12566 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:55.320654  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:55.322444  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:04:55.328140  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:55.323609  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:04:55.335122  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 249524us
I20260709 13:04:55.337450 12640 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:55.337802 12167 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:55.337972 12640 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:55.338279 12167 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:55.338482 12640 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:55.338743 12167 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:55.323653 12550 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 13:04:55.328775  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.341306  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:55.326539  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:55.296932 12543 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.345319  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:55.348474  9194 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:55.344269  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:04:55.360340  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 54160.8us
I20260709 13:04:55.360994 12493 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:55.362099 12550 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:55.362579 12550 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:55.362793  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:55.362977 12550 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:55.363953 12291 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:55.364360 12291 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:55.364470  9190 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:55.364756 12291 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:55.365520  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:55.334743 12618 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:04:55.371412  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:55.376840 12529 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:55.377389 12529 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:55.377867 12529 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:55.347949 12567 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:04:55.380875  9119 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:04:55.382910  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.383876  9114 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:55.385577  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.389218 12668 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:55.390267 12566 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:55.352138 12612 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:55.392562  9122 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:55.392262  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 987) took 4430 ms. Trace:
W20260709 13:04:55.396559  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 70077.8us
W20260709 13:04:55.392832 12706 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:04:55.389799 12668 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:55.408298  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 100777us
I20260709 13:04:55.394907  9860 rpcz_store.cc:276] 0709 13:04:50.962324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:52.774773 (+1812449us) service_pool.cc:224] Handling call
0709 13:04:52.775107 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.531133 (+1756026us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:04:54.762359 (+231226us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.289558 (+527199us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:55.289603 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:55.289646 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.391885 (+102239us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54004a3dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:55.391929 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.392101 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1513,"prepare.queue_time_us":249910,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":33,"raft.queue_time_us":8842,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4945,"spinlock_wait_cycles":4037505280,"thread_start_us":700671,"threads_started":2}
I20260709 13:04:55.429576  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:55.433248  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:55.153600  9269 rpcz_store.cc:276] 0709 13:04:52.616250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.544216 (+1927966us) service_pool.cc:224] Handling call
0709 13:04:54.544638 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.545051 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:54.769434 (+224383us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.054496 (+285062us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:55.054584 (+    88us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:55.054652 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.054761 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.054939 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27207,"prepare.queue_time_us":202728,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":3320,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":22451,"thread_start_us":485635,"threads_started":2}
I20260709 13:04:55.446305  9861 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:55.450294  9191 raft_consensus.cc:3055] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:55.450752  9191 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:55.451481  9194 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:55.391371 12550 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:55.455974  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 132430us
W20260709 13:04:55.458092 12497 consensus_meta.cc:287] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.005s
I20260709 13:04:55.501228  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:55.505973  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 43953.1us
I20260709 13:04:55.463891 12543 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:55.516063 12609 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:04:55.517042  9191 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.518121 12685 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:55.524266 12355 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Committing config change with OpId 8.6: config changed from index -1 to 6, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:04:55.516050  9271 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:04:55.443838 12605 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:55.533002  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 101920us
W20260709 13:04:55.527842 12568 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.068s	user 0.007s	sys 0.011s
I20260709 13:04:55.539976  9271 pending_rounds.cc:85] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 4
I20260709 13:04:55.548879  9271 pending_rounds.cc:107] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:04:55.549247  9271 raft_consensus.cc:2889] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:55.539103 12609 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:55.553382 12568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.008s	sys 0.011s
I20260709 13:04:55.538947 12685 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:55.570953  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:55.567471  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 256110us
I20260709 13:04:55.512110 12386 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:04:55.574429  9191 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:55.570429 12609 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:55.590183  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.578831 12497 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
W20260709 13:04:55.602344  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.554456 12674 consensus_queue.cc:1059] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:04:55.580241 12685 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:55.609505  9191 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:04:55.611734 12529 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.613688  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 997) took 1062 ms. Trace:
I20260709 13:04:55.635468  9194 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
W20260709 13:04:55.636198  9826 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.857s (SENT)
I20260709 13:04:55.600474 12612 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:04:55.641798  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:55.589766 12692 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:04:55.632372  9855 rpcz_store.cc:276] 0709 13:04:54.547820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:54.833077 (+285257us) service_pool.cc:224] Handling call
0709 13:04:54.839657 (+  6580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:54.993960 (+154303us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:04:55.106875 (+112915us) log.cc:844] Serialized 14 byte log entry
0709 13:04:55.609451 (+502576us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:55.609503 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:55.609541 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.609674 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:55.609770 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":126554,"prepare.run_cpu_time_us":6833,"prepare.run_wall_time_us":9737,"raft.queue_time_us":10518,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":3269,"thread_start_us":392389,"threads_started":1}
I20260709 13:04:55.643509 12612 consensus_queue.cc:1243] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:04:55.647235 12583 raft_consensus.cc:3055] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:55.647985 12543 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:04:55.658645 12710 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:04:55.655224 12550 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:55.678510  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.679452  9118 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:55.680797  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.681708  9120 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:55.683314  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.684643  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.685458  9119 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:55.686304  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.687155  9123 raft_consensus.cc:2410] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:04:55.687920 12545 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:55.688066  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.688884  9115 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:55.688498 12545 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.690090  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.674606 12291 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.694108 12291 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:55.684688  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.683414  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.666134 12583 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:55.696146  9122 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:55.697105  9116 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:04:55.701804  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.702514  9118 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:55.702348 12167 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.704685 12605 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:55.726994 12291 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:04:55.747165 12545 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:55.795951 12709 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.342s	user 0.002s	sys 0.002s
W20260709 13:04:55.799345  9087 connection.cc:256] RPC call timeout handler was delayed by -3.6541306279999617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 13:04:55.799939  9087 connection.cc:256] RPC call timeout handler was delayed by -3.5880842009999583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
W20260709 13:04:55.800354  9087 connection.cc:256] RPC call timeout handler was delayed by -3.5538355649999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 13:04:55.800738  9087 connection.cc:256] RPC call timeout handler was delayed by -3.4717567149999695s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21789999999999998s for a response
W20260709 13:04:55.801147  9087 connection.cc:256] RPC call timeout handler was delayed by -3.4242355650000036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
W20260709 13:04:55.801538  9087 connection.cc:256] RPC call timeout handler was delayed by -3.3679306279999537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
I20260709 13:04:55.758538 12366 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:55.734397 12640 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:55.801972  9087 connection.cc:256] RPC call timeout handler was delayed by -3.2585554249999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.215s for a response
W20260709 13:04:55.821592  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.823037  9117 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:55.812783 12291 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:04:55.827759  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 300089us
I20260709 13:04:55.819624 12167 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:55.829694  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 263839us
W20260709 13:04:55.830552  9087 connection.cc:256] RPC call timeout handler was delayed by -3.2350247059999333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 13:04:55.831441  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 239043us
W20260709 13:04:55.844982  9087 connection.cc:256] RPC call timeout handler was delayed by -3.1600204849999614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
I20260709 13:04:55.824126 12529 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:55.797683 12583 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:55.847208  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:55.851578  9861 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
W20260709 13:04:55.849937  9087 connection.cc:256] RPC call timeout handler was delayed by -2.5442027719999487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 13:04:55.850718  9855 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: []
W20260709 13:04:55.852787  9087 connection.cc:256] RPC call timeout handler was delayed by -2.4515027719999125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:04:55.853291  9087 connection.cc:256] RPC call timeout handler was delayed by -2.0294027719999121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21050000000000002s for a response
W20260709 13:04:55.853696  9087 connection.cc:256] RPC call timeout handler was delayed by -1.9700027719999298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
W20260709 13:04:55.850742 12719 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:04:55.856401  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:55.859788  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.843137 12291 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:55.821540 12366 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.869765  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.869951  9862 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:55.866992  9164 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:55.869565  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.878863  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.880726  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.886974  9856 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:55.887778  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.884465  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.890865  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.862747 12721 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:55.895630  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:55.897253  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.900380  9853 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:55.898085  9123 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:55.901407  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.901278 12721 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:55.901840  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.902070  9118 raft_consensus.cc:2376] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5. Re-sending same reply.
I20260709 13:04:55.902575  9123 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:55.904668  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.905567  9116 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:55.906291 12721 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:55.860692 12423 consensus_queue.cc:1059] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:04:55.907635 12291 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:55.908561  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.909072  9115 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:04:55.909214 12291 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:55.909687  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.910718  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.908417 12423 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:55.911860  9122 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:55.911911  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:55.912565 12714 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:04:55.913350  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.913264  9114 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:55.914165  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.914721  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.913251 12423 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:55.915041 12593 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:55.915337  9116 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:55.915500 12593 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:55.916042  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.916483  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.916553 12593 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.917388 12567 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:55.917510  9118 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:04:55.917971 12567 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:55.918512  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.919235  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.919719  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:55.920416 12567 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:55.921031  9118 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:55.921725  9121 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:55.921648  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.922648  9854 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:04:55.924249  9087 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.533s (SENT)
W20260709 13:04:55.925366  9087 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.609s (SENT)
I20260709 13:04:55.925805  9087 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:55.928342  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:55.928997  9118 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
W20260709 13:04:55.931757  9087 leader_election.cc:336] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.636s (SENT)
I20260709 13:04:55.932279  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:55.928388  9089 leader_election.cc:400] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:04:55.934218  9089 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:55.936259  9115 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
W20260709 13:04:55.937889  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:55.939021  9119 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:55.940091  9122 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:55.940231 12692 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:55.943146 12692 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:55.943678 12692 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:55.944753 12692 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:55.947383 12631 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:55.860239 12386 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:04:55.872051 12529 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.084s
I20260709 13:04:55.959470 12382 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:55.960096  9859 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:55.961776 12366 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:55.885349 12583 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:55.971765  9119 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
W20260709 13:04:55.975244 12668 consensus_meta.cc:287] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.003s
I20260709 13:04:55.987126  9862 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:55.987565  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:55.987701  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 363333us
I20260709 13:04:55.918435 12593 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:55.989595 12382 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:55.985841 12631 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:56.004010 12529 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:55.994258 12474 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:55.992288 12721 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:56.017875  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:56.018414 12529 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:55.999327 12668 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:56.032356  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 183398us
I20260709 13:04:56.034382  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:56.035784  9235 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:56.115406  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 265828us
W20260709 13:04:56.118249  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 251348us
I20260709 13:04:56.119279  9164 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:56.147256 12723 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:56.147905 12723 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.148375 12723 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:56.117671 12474 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.152444 12730 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:56.154337 12474 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:56.114614 12167 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:56.155269 12167 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.156260 12631 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.157729 12631 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.157917 12167 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:56.160164 12685 consensus_meta.cc:287] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.003s
I20260709 13:04:56.160836 12685 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.165424 12550 consensus_queue.cc:1059] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:56.165660 12692 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:56.166570 12550 raft_consensus.cc:3055] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:56.166461 12692 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:56.166953 12550 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:56.167272 12692 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:56.167886 12568 raft_consensus.cc:3055] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:56.168277 12568 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:56.168879 12742 consensus_queue.cc:1059] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:56.171161  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:56.177678  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:04:56.190162  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.192072  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:56.193080  9119 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:56.223073  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 281256us
I20260709 13:04:56.167603 12692 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:04:56.241842  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 53627.3us
I20260709 13:04:56.169209 12568 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.145658 12725 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:56.253340  9825 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:56.243973  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 97143.7us
I20260709 13:04:56.244757  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:56.248517  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.170472 12726 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:56.259076  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 95328us
I20260709 13:04:56.261303  9121 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:56.236282 12618 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:04:56.265936  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.314716  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 60047.1us
I20260709 13:04:56.183018 12605 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
W20260709 13:04:56.312588  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 53303.4us
W20260709 13:04:56.310878 12719 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:04:56.260634 12726 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:56.327349  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 173620us
I20260709 13:04:56.335158  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:56.336088  9862 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:56.325201 12568 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:56.327491 12726 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:56.258914 12550 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.314267 12725 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.310626 12529 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:56.259176 12382 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.358433  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:56.365144  9087 leader_election.cc:336] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.127s (SENT)
W20260709 13:04:56.364328 12609 consensus_meta.cc:287] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.750s	user 0.011s	sys 0.001s
I20260709 13:04:56.359244 12618 raft_consensus.cc:697] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:56.368252 12727 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:04:56.382108 12668 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:56.382710 12382 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.379978 12609 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:56.384541 12735 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:04:56.385334 12717 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.703s	user 0.009s	sys 0.001s
I20260709 13:04:56.330528 12692 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.403832 12692 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.394374 12749 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:04:56.408078 12749 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:56.408496 12749 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:56.409093 12718 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.725s	user 0.002s	sys 0.002s
I20260709 13:04:56.409925  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2341) took 1218 ms. Trace:
I20260709 13:04:56.410285  9268 rpcz_store.cc:276] 0709 13:04:55.191198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.195022 (+  3824us) service_pool.cc:224] Handling call
0709 13:04:55.195346 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:55.198650 (+  3304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:55.198680 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:55.215873 (+ 17193us) log.cc:844] Serialized 26 byte log entry
0709 13:04:55.534142 (+318269us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:04:55.534178 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:55.534260 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:56.366503 (+832243us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:56.409625 (+ 43122us) raft_consensus.cc:1685] finished
0709 13:04:56.409644 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.409848 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1066,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"spinlock_wait_cycles":19221248,"thread_start_us":317865,"threads_started":1,"wal-append.queue_time_us":466379}
W20260709 13:04:56.394400  9087 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.150s (SENT)
I20260709 13:04:56.382809 12550 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:56.413470  9087 leader_election.cc:336] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.177s (SENT)
W20260709 13:04:56.413630 12716 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.661s	user 0.002s	sys 0.000s
I20260709 13:04:56.413887  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:56.394984  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:56.397369 12739 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:04:56.416312 12748 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:56.417232 12685 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:56.397264 12752 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.418426 12752 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.401348 12609 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:56.421248 12752 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:56.381357 12731 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:56.422660 12612 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:56.422461 12723 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.423379 12640 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:56.421772  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.427105 12711 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.908s	user 0.000s	sys 0.005s
I20260709 13:04:56.429821  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:56.430482  9119 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:04:56.433279 12723 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:56.433389 12291 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.000s
I20260709 13:04:56.434324  9235 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:56.435298  9235 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:56.435534 12751 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:56.436538 12752 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:56.437319 12752 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:56.438513 12752 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.439991 12529 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:56.440192  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:56.391242 12727 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.461001  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:56.466797  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 103473us
I20260709 13:04:56.422508 12731 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:56.463510 12355 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:38634 (request call id 19) took 9359 ms. Trace:
I20260709 13:04:56.467161 12751 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:04:56.468173 12727 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:56.476016  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:56.479159  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:56.481523  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:56.482081  9862 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:56.486130 12731 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.496181  9862 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
W20260709 13:04:56.505584 12593 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.001s
I20260709 13:04:56.507836 12668 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:56.508440 12668 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.508925 12668 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:56.511870  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.511651 12751 raft_consensus.cc:697] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:56.513370 12751 consensus_queue.cc:237] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:56.520576  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 169408us
I20260709 13:04:56.518147  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2030) took 1072 ms. Trace:
I20260709 13:04:56.468845 12749 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:56.491533 12750 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:56.521840  9190 rpcz_store.cc:276] 0709 13:04:55.445424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:55.445607 (+   183us) service_pool.cc:224] Handling call
0709 13:04:55.486415 (+ 40808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:55.519391 (+ 32976us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:04:55.874374 (+354983us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.244374 (+370000us) log.cc:844] Serialized 14 byte log entry
0709 13:04:56.401200 (+156826us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b180035c620. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535c52a6 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1
0709 13:04:56.517660 (+116460us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b180035c620. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f7653571c0a 00007f765ccb1219 00007f76535f9a40 00007f76535f44e4 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287 00007f76566ff208
0709 13:04:56.517820 (+   160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:56.517866 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:04:56.517913 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:56.517975 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:56.518085 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51589,"prepare.queue_time_us":486170,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":78,"raft.queue_time_us":328051,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":3290,"spinlock_wait_cycles":92653824,"thread_start_us":659674,"threads_started":2,"wal-append.queue_time_us":497560}
I20260709 13:04:56.537343  9114 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:56.544054 12497 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:04:56.542749 12750 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:56.477595 12355 rpcz_store.cc:276] 0709 13:04:47.032442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.032596 (+   154us) service_pool.cc:224] Handling call
0709 13:04:47.402409 (+369813us) log.cc:844] Serialized 499 byte log entry
0709 13:04:48.043877 (+641468us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54002a3d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566ea735 00007f765b6aad29 00007f7654c291d4 00007f7654c29162 00007f7654c29091 00007f7654c2900c 00007f7654c282c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:56.379032 (+8335155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1171815,"raft.queue_time_us":1174619,"raft.run_cpu_time_us":490,"raft.run_wall_time_us":489,"spinlock_wait_cycles":428020608}
I20260709 13:04:56.560899 12725 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:56.561816 12750 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.499640 12618 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:56.578248  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 257052us
W20260709 13:04:56.588317 12762 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:04:56.589211 12747 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:56.589811 12747 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:56.590205 12355 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:04:56.590602 12355 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:56.590991 12747 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:56.593721 12747 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:56.595986 12748 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:56.596835  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 284838us
I20260709 13:04:56.602388 12749 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:56.602855  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 360344us
I20260709 13:04:56.606124  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:56.606428 12545 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:56.606768  9116 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:56.603518 12745 raft_consensus.cc:3055] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:56.609807 12366 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:56.611665 12745 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:56.626223 12745 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.591467 12355 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:56.614715 12366 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.631706 12366 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:56.591652 12726 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.641220 12668 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.642360 12366 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.644810 12567 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:56.646163 12543 raft_consensus.cc:3055] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:04:56.646552 12543 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:56.647459 12543 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.649278 12543 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:04:56.653034  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.655088  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 51179.2us
I20260709 13:04:56.658215 12726 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:56.661151 12764 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:56.663101  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:56.664084 12725 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:04:56.664444 12745 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:56.665169  9161 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:56.664500 12725 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:56.666985 12725 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:56.669793 12727 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.666466 12545 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:56.671813  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.672151  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:56.672070 12545 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.672811  9114 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:04:56.673046  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:56.676481  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 95886.1us
I20260709 13:04:56.677242  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:56.675472  9825 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:56.681598  9862 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:04:56.697453  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:56.698436  9855 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 3. Current term is 4. Ops: []
I20260709 13:04:56.701056  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:56.704015  9264 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:56.706067 12545 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:56.706275 12638 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:04:56.706724  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:56.707288  9268 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.693966 12566 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:56.708070 12566 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:56.708453 12566 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:56.712003  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.721230  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:56.721849  9272 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:56.724133  9114 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:04:56.725448  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:56.726428  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.727391 12749 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:04:56.727947 12749 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:56.728828 12749 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.730460 12382 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:56.730624  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 53451.4us
I20260709 13:04:56.730875 12382 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:56.731258 12382 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:56.706377 12674 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:56.732525  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:56.733481 12674 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:56.735054 12674 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:04:56.737643 12668 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:56.737738 12366 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:56.744590  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 47235.2us
I20260709 13:04:56.760867  9161 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:56.747876  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.761353  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:56.761826 12725 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.765302  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:56.735823  9271 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:56.755877 12727 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:56.766202 12583 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:56.766623 12583 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:56.766752 12727 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:56.757035 12733 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:56.768020 12733 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:56.767014 12583 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:56.768354  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:56.769238 12726 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:04:56.770481  9161 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:56.758006  9018 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 5 to 10, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 10 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:56.772130  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:56.727118  9264 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:04:56.771538  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:56.775959  9116 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:56.777601  9268 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:56.783393  9272 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:56.786558 12567 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:56.787288 12567 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:56.787871 12567 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:56.783795  9235 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:56.848374  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 167883us
W20260709 13:04:56.857942  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 85896.2us
I20260709 13:04:56.850237  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:56.783610 12726 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:56.866953  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:56.831346  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.868928  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:56.871444  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:56.871840  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:56.872223  9855 raft_consensus.cc:2368] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10.
I20260709 13:04:56.872534  9191 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:56.807006 12386 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:56.874097  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:56.875558  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:56.875978  9853 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:04:56.876211  9114 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:56.863976 12751 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:04:56.880160  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:56.880967  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:56.878459  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:56.877804  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:56.849081 12725 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:56.911666  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.819633 12566 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.958127  9114 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:56.933737 12726 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:04:56.933881 12776 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:56.934669 12355 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.874046 12423 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.984447  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:56.985394  9862 raft_consensus.cc:2410] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:04:56.864094 12593 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
W20260709 13:04:56.950784  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 100624us
I20260709 13:04:56.951364  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:56.953271 12609 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:04:56.956151  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:56.990272  9194 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:56.869607 12764 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:56.881600  9190 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:56.959570  9854 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:04:56.963176  9121 raft_consensus.cc:2410] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:04:56.963562  9860 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:56.987124  9855 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
W20260709 13:04:56.974226  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:56.987865  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:56.958323 12753 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:04:56.995399  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:56.993485  9862 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:04:56.949481 12751 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:04:56.998195  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:57.006717  9854 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:56.999184  9272 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:56.999698  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:57.000851  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:57.001937  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:56.994163 12423 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:56.983554 12543 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:56.984019 12382 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.003970  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.003916  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.005134  9189 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:57.005686  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.006477  9825 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:57.013329  9192 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:57.013408 12727 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:04:57.014235  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:57.015010  9856 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:04:57.014925 12751 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:57.016028  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.016644  9198 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:57.017702  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:56.949246 12733 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.989693 12726 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:57.021341 12726 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:56.951432 12583 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.012306  9860 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:04:57.012172  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 145292us
I20260709 13:04:56.972885 12618 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:57.014285  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:57.008972  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:56.989759 12609 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:57.046711  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.065102  9193 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:04:57.065630  9853 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:56.998251 12764 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:57.067341  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 60985us
I20260709 13:04:57.028342  9018 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:04:57.067772  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:57.067811  9089 leader_election.cc:336] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.171s (SENT)
I20260709 13:04:57.068305  9190 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:04:57.074656  9854 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:57.133271  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.133665  9195 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 6. Current term is 7. Ops: []
W20260709 13:04:57.134142  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 108083us
W20260709 13:04:57.137537  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 62031.7us
W20260709 13:04:57.139350  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.014292 12674 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:04:57.140646  9826 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.662s (SENT)
I20260709 13:04:57.025738 12727 raft_consensus.cc:697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:56.989296 12776 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.067060 12609 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:57.145329  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.148013  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:57.149469  9194 raft_consensus.cc:2410] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:04:57.149963  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.150162  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:57.147682  9826 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.694s (SENT)
I20260709 13:04:57.152119  9196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:04:57.152733  9196 raft_consensus.cc:606] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
W20260709 13:04:57.152920  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.153312  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.077916 12366 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:57.153837  9115 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:57.160250  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:57.151638  9826 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:57.144124  9862 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:04:57.160250  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:57.161798  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:57.162503  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 97220.1us
I20260709 13:04:57.164151 12423 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:57.164822 12566 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:57.165668 12775 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:04:57.166023 12366 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:57.165778 12567 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.166776  9854 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:04:57.062060 12543 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.168114  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:57.068202 12618 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:57.172910 12382 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:57.145223 12779 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:04:57.162127  9121 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:04:57.267339  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 119695us
I20260709 13:04:57.283032  9123 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:57.179665  9120 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:04:57.230129  9853 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:04:57.282856  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.240020  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:57.262359  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.183072  9861 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:57.289597  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.143456 12674 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:57.309234  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:57.313009  9855 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:57.303633  9272 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:57.314275  9862 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:57.231616 12631 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:04:57.179993 11805 consensus_queue.cc:1059] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.145s
W20260709 13:04:57.324608 12778 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 13:04:57.324816 12764 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:57.324893  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:57.325650  9853 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:57.326412 12366 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.326763 12724 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:04:57.285621 12747 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:57.329712 12747 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.330156 12543 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:57.255712 12781 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:57.338932  9853 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:04:57.350837 12423 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:57.351454 12423 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:57.359313  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.362152  9190 raft_consensus.cc:2410] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:04:57.331616 12747 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:57.337412 12727 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.384112 12756 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:04:57.368711 12776 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:57.372373  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:57.373312  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:57.381623  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.394348  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:57.395058  9855 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:57.403093  9853 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:57.396589 12760 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.413456  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.415027  9190 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:57.418650  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:57.427899  9859 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:57.436299  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.410674 12747 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:57.451828 12747 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:57.454216 12747 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:04:57.421538 12750 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:57.456224 12750 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:57.457227  9087 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.999s (SENT)
I20260709 13:04:57.457846  9087 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:57.458621 12750 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:57.441345 12749 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.459589 12749 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:57.461802 12749 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:57.441792 12725 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:04:57.462539 12725 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:57.463449 12725 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:57.443073  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.466279  9190 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:04:57.443871  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.467530  9267 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:57.469720  9828 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:57.429229 12760 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.465094 12776 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.428633 12583 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:57.474231  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.494163  9268 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:57.473325  9828 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:57.494431  9193 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:57.495400  9828 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:57.495618 12781 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:04:57.496892 12566 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:57.497203  9828 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:57.497300 12566 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:57.498282 12566 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.499586 12566 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:57.492726 12776 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:57.520852  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 204930us
W20260709 13:04:57.500118  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.508466  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:57.517382  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.518397  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.522897  9267 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:57.523100  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:57.525833  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:57.411991 12674 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.531734  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.500758 12545 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:04:57.532302  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 211468us
W20260709 13:04:57.550602  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.524521 12583 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:04:57.562147  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.528542 12769 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:57.592137  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:57.594301  9860 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:04:57.598484  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.599324  9268 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:57.543736 12386 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:57.545199 12545 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:57.608134  9267 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:04:57.591517 12674 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:57.470934 12423 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.600843 12605 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:57.713383  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:57.608896 12545 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:57.753618  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.757009  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.758868  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:57.601416  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 235180us
I20260709 13:04:57.720921 12747 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.765412  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:57.768255  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:57.774379  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:04:57.781622 12790 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:04:57.749861 12756 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.771473 12423 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:04:57.784189 12756 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:57.786378  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:57.975720  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.786514  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:58.192981  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:57.839696  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:58.254854  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.015s	sys 0.019s
W20260709 13:04:58.255395  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.015s	sys 0.019s
I20260709 13:04:58.255597  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:57.833374  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:57.929173  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.256855  9263 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:58.257508  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:57.839819  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.038491  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.261334  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.057687  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.928560  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:57.934394  9089 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.725s (SENT)
I20260709 13:04:58.264590  9264 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:58.264861  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 330512us
I20260709 13:04:58.258714  9118 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:04:57.871726  9826 leader_election.cc:336] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.537s (SENT)
W20260709 13:04:58.271505  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 399816us
W20260709 13:04:58.272370  9826 leader_election.cc:336] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.224s (SENT)
I20260709 13:04:58.272683  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:58.060030  9190 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
W20260709 13:04:58.260195  9859 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.004s	sys 0.058s
I20260709 13:04:58.058081  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:04:57.967944  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 442157us
I20260709 13:04:58.104605  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:58.046030  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 270182us
W20260709 13:04:58.279539  9859 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.005s	sys 0.058s
W20260709 13:04:58.255769  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 693305us
I20260709 13:04:57.755136 12618 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:58.277164  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.104138  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.283492  9195 raft_consensus.cc:2410] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:04:58.263563  9192 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:58.270114  9196 raft_consensus.cc:2368] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10.
I20260709 13:04:58.271613  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:58.276706  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 819491us
W20260709 13:04:57.865733 12609 consensus_meta.cc:287] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.644s	user 0.004s	sys 0.004s
I20260709 13:04:57.786624  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:58.287703  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.280401  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:58.288329  9115 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:04:58.287820  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.284425  9265 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:58.279084  9271 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:04:58.282824  9263 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:04:58.289521  9828 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:58.252076 12474 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:57.842301 12760 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:57.859140 12796 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:57.911046 12805 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.292517  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.259682 12756 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:58.301980  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:58.306082  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:57.841672 12794 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:57.847834 12795 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.320451  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.322039  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.859318 12802 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.309304  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:57.954056 12605 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:57.864580 12776 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:58.312017  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:58.312228  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.315316  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.315464  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.313824  9197 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:58.316135  9264 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
W20260709 13:04:58.316466  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44125.4us
I20260709 13:04:57.777695 12747 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:58.327200  9860 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.732s	user 0.002s	sys 0.007s
I20260709 13:04:58.321617  9265 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:58.323338  9271 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 4.
I20260709 13:04:58.292263 12618 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.330899  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:58.304936 12609 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:58.331661  9857 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:04:58.406399  9018 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 2 to 4, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:04:58.319994 12805 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:58.425220  9118 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:04:58.425746  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 611) took 1422 ms. Trace:
I20260709 13:04:58.432525  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:58.360325  9858 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.037s	sys 0.045s
I20260709 13:04:58.442741  9268 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
W20260709 13:04:58.442863  9858 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.054s	sys 0.074s
I20260709 13:04:58.442759  9194 rpcz_store.cc:276] 0709 13:04:57.003207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.003707 (+   500us) service_pool.cc:224] Handling call
0709 13:04:58.425541 (+1421834us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540010cf54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:58.425676 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313398400}
I20260709 13:04:58.439184  9860 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:04:58.439289  9267 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:04:58.442287  9826 leader_election.cc:336] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.242s (SENT)
I20260709 13:04:58.444875  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:58.443372  9190 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:04:58.445463  9263 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:04:58.456571  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:58.337304  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:58.460664  9121 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:58.426373 12800 ts_tablet_manager.cc:933] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:58.460867  9120 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
W20260709 13:04:58.466437  9087 leader_election.cc:336] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.681s (SENT)
I20260709 13:04:58.466919  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:58.467298  9191 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:04:58.407249 12423 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:58.468794  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1009) took 1473 ms. Trace:
W20260709 13:04:58.480747  9826 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.590s (SENT)
I20260709 13:04:58.481765  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:58.410565 12815 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:58.336366 12807 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.440486 12769 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:58.457221 12759 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.481385  9858 rpcz_store.cc:276] 0709 13:04:56.991230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.991352 (+   122us) service_pool.cc:224] Handling call
0709 13:04:56.991628 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:56.991849 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:04:57.393997 (+402148us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.446161 (+1052164us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.464246 (+ 18085us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:58.464307 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:04:58.464357 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.464429 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.464542 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17883,"prepare.queue_time_us":141387,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":92,"raft.queue_time_us":534113,"spinlock_wait_cycles":19040640,"thread_start_us":1407020,"threads_started":3,"wal-append.queue_time_us":852707}
I20260709 13:04:58.491065  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:58.494832 12803 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:04:58.469755 12781 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:58.463325 12809 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:58.514246  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 241408us
W20260709 13:04:58.498647 12566 consensus_meta.cc:287] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.974s	user 0.010s	sys 0.001s
I20260709 13:04:58.494582 12817 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:58.501247  9264 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:58.443871 12605 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:58.518468 12776 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:58.320286 12796 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:58.335925 12795 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:58.516460  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.516948  9267 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:04:58.410275 12794 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:04:58.520188 12809 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:58.520659  9826 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.656s (SENT)
I20260709 13:04:58.545070  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:58.441753 12618 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:58.524763  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:58.484692 12798 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:58.523483 12605 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:58.552198  9120 raft_consensus.cc:2410] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:04:58.481200 12423 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:58.552901 12423 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:58.553197 12780 raft_consensus.cc:3055] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:04:58.553689 12780 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:58.554627 12780 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:58.556486 12609 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:58.557545  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.557914  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.560153  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.362970 12802 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:04:58.538190  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.538915 12545 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.782s	user 0.007s	sys 0.002s
W20260709 13:04:58.539446  9855 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 937) took 1544 ms (client timeout 2047 ms). Trace:
W20260709 13:04:58.614624  9855 rpcz_store.cc:269] 0709 13:04:56.994691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.994813 (+   122us) service_pool.cc:224] Handling call
0709 13:04:58.539276 (+1544463us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b5400671e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:58.539386 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":701034880}
I20260709 13:04:58.496094 12811 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:04:58.542683  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.550536 12809 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:58.551028 12798 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:58.581214  9862 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
W20260709 13:04:58.534869  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 245456us
I20260709 13:04:58.609206  9198 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:04:58.620543  9828 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:58.622956  9195 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:58.623910  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:58.629567  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.513025 12759 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:58.633023  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:58.639739  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.640328  9267 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:04:58.640828  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 120215us
W20260709 13:04:58.641769  9826 leader_election.cc:336] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.748s (SENT)
W20260709 13:04:58.642907  9826 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.874s (SENT)
W20260709 13:04:58.644272  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.646860  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:58.643457  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:58.497668 12807 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:58.621763 12545 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:58.637588 12820 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:58.632355 12751 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.660205  9119 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:04:58.657675  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:58.664561  9269 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
W20260709 13:04:58.666327  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.668318  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:04:58.663555 12820 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:58.673944  9857 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:04:58.674085  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.622546 12605 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:04:58.653251 12815 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:58.686443  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.688323  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:58.689465  9858 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
W20260709 13:04:58.691038  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 224623us
I20260709 13:04:58.691891  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.657869 12814 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:58.693440 12814 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:58.693830 12814 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:58.686208 12543 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:04:58.696579 12674 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.981s	user 0.011s	sys 0.000s
W20260709 13:04:58.697944 12806 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.206s	user 0.003s	sys 0.000s
W20260709 13:04:58.699563  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:58.699940  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.700536  9265 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:04:58.702426 12798 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:58.703706 12795 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:58.705304 12796 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:58.705457  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.690969 12794 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:04:58.688277 12800 tablet_copy_client.cc:323] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:04:58.708171  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:04:58.733891  9089 leader_election.cc:336] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.926s (SENT)
W20260709 13:04:58.800495  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 66674.7us
I20260709 13:04:58.695351  9189 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:04:58.689465  9193 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:58.704850  9018 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.706250  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1012) took 1150 ms. Trace:
W20260709 13:04:58.725275  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.750564  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 63185.5us
I20260709 13:04:58.805505  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:58.656980 12819 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:04:58.832934 12819 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:58.687130 12618 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:58.752453 12811 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:04:58.836599 12811 raft_consensus.cc:3055] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:58.836956 12811 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:58.802196  9859 rpcz_store.cc:276] 0709 13:04:57.555424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.557649 (+  2225us) service_pool.cc:224] Handling call
0709 13:04:57.596815 (+ 39166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:58.281180 (+684365us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:58.330341 (+ 49161us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.649193 (+318852us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.649681 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.649818 (+   137us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:04:58.650032 (+   214us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.650235 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.650316 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.705987 (+ 55671us) raft_consensus.cc:1685] finished
0709 13:04:58.706008 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.706180 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31102,"prepare.queue_time_us":39035,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":14018,"raft.queue_time_us":360721,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":34878,"spinlock_wait_cycles":12926080,"thread_start_us":806051,"threads_started":2,"wal-append.queue_time_us":184778}
I20260709 13:04:58.837878 12811 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:58.839131 12811 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:04:58.691912  9087 leader_election.cc:336] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.720s (SENT)
I20260709 13:04:58.796046 12567 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:58.796306 12823 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:58.827755 12167 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:04:58.873224  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:58.856348  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 191455us
I20260709 13:04:58.751426  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 947) took 1380 ms. Trace:
I20260709 13:04:58.755908 12747 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:58.861943  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:58.756155  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.839587 12781 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:04:58.840284 12545 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:58.762112 12423 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:58.686975 12751 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:04:59.055511  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.112439  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:59.018334  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.096256  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:59.054440  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.121523  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:59.070866  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.046022  9192 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:59.109889  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:58.884490  9856 rpcz_store.cc:276] 0709 13:04:57.370867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.406040 (+ 35173us) service_pool.cc:224] Handling call
0709 13:04:57.424152 (+ 18112us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:57.431702 (+  7550us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:57.441454 (+  9752us) log.cc:844] Serialized 14 byte log entry
0709 13:04:58.212157 (+770703us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:58.238842 (+ 26685us) log.cc:844] Serialized 26 byte log entry
0709 13:04:58.239009 (+   167us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:58.239058 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:58.239150 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:58.697177 (+458027us) spinlock_profiling.cc:243] Waited 9.52 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:58.706198 (+  9021us) raft_consensus.cc:1685] finished
0709 13:04:58.751167 (+ 44969us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:58.751361 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17110,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":518994,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":19772,"spinlock_wait_cycles":21899648,"thread_start_us":736840,"threads_started":2,"wal-append.queue_time_us":991974}
I20260709 13:04:59.054958  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.229537  9193 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:59.059628  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 195866us
I20260709 13:04:59.133908  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.080273  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1014) took 1521 ms. Trace:
I20260709 13:04:59.071867  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.234642  9862 rpcz_store.cc:276] 0709 13:04:57.560341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.786237 (+225896us) service_pool.cc:224] Handling call
0709 13:04:59.080122 (+1293885us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400733054. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:59.080219 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621251328}
I20260709 13:04:59.228369  9263 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:59.049307  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.089730  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.103372  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.236743  9270 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:04:58.885052  9826 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.343s (SENT)
W20260709 13:04:59.238390  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 546454us
I20260709 13:04:59.238538  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.056802  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:59.226047  9271 raft_consensus.cc:2376] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7. Re-sending same reply.
I20260709 13:04:58.869462 12815 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.241936  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 752) took 1132 ms. Trace:
I20260709 13:04:59.249571  9270 rpcz_store.cc:276] 0709 13:04:58.109936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.110090 (+   154us) service_pool.cc:224] Handling call
0709 13:04:59.089515 (+979425us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:59.241861 (+152346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7214336}
I20260709 13:04:59.243181  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:59.244783  9087 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.753s (SENT)
I20260709 13:04:59.250272  9853 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:59.245476  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:59.081586 12725 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:04:59.247244  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 190533us
I20260709 13:04:59.248245  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.244592  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.241802  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:59.250160  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 365149us
I20260709 13:04:59.258728  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.869730 12167 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.251384  9193 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:59.060230 12816 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.594s	user 0.002s	sys 0.000s
W20260709 13:04:59.268344  9826 leader_election.cc:336] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.900s (SENT)
I20260709 13:04:59.251616  9272 raft_consensus.cc:2376] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8. Re-sending same reply.
I20260709 13:04:59.269142  9826 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:59.252262  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:04:59.251614  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.048018 12821 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.099022 12792 raft_consensus.cc:3055] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:04:59.254776  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:58.923005 12776 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:58.863286 12819 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:59.261248  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.274322 12167 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:59.301389  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.262009  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.244992 12755 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:04:59.270722  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.270988  9269 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: []
W20260709 13:04:59.273051  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.320039  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.319331  9193 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:04:59.252264 12831 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:04:58.924765 12567 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:59.321271  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:59.321841  9193 raft_consensus.cc:2376] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6. Re-sending same reply.
I20260709 13:04:59.321475 12567 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.322685  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:59.258136 12545 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:59.271880  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.325083  9271 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:59.326164  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.326874  9271 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:59.327965  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.277285  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.331825 12759 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:59.332883 12802 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:59.115792 12814 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.333030  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.333841  9198 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:04:59.334951 12814 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:04:59.336314  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2023) took 1053 ms. Trace:
I20260709 13:04:59.336670  9189 rpcz_store.cc:276] 0709 13:04:58.282705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.301697 (+ 18992us) service_pool.cc:224] Handling call
0709 13:04:59.336152 (+1034455us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b540010ca54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:59.336242 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564025728}
I20260709 13:04:59.337611  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.282146 12792 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:04:59.283552 12794 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:59.067446 12474 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:04:59.285890  9265 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:04:59.074847 12820 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.315200  9195 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
W20260709 13:04:59.273324  9087 leader_election.cc:336] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.999s (SENT)
I20260709 13:04:59.334892  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:58.871526 12823 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:59.371439 12823 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:59.279203  9826 leader_election.cc:336] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.929s (SENT)
I20260709 13:04:59.372175  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:04:59.373091  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 93931.4us
W20260709 13:04:59.373711  9826 leader_election.cc:336] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.937s (SENT)
I20260709 13:04:59.374066  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:59.374174 12811 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:59.375097 12811 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:59.287606 12821 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:59.255797 12815 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.338654  9861 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:04:59.360330 12751 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:04:59.384418  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.025s	sys 0.048s
W20260709 13:04:59.384723  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.025s	sys 0.048s
I20260709 13:04:59.356695 12831 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:59.361847 12423 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.313139 12609 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:04:59.363876 12792 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.366374  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.384058 12567 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.368208  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:59.295933 12776 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.376227  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.379300 12794 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.382838  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 607) took 2501 ms. Trace:
W20260709 13:04:59.386181  9194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.013s	sys 0.059s
W20260709 13:04:59.393119  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 710373us
I20260709 13:04:59.383975 12807 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.405848  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.427013  9195 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:59.368702 12474 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:59.407675  9193 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:59.409433  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.437985  9270 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:59.367046  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.413631  9265 raft_consensus.cc:2410] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:59.439720  9271 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:59.057157 12833 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.207s
W20260709 13:04:59.417148  9194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.013s	sys 0.060s
W20260709 13:04:59.416916  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.445847  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.446203  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.448101  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:59.448241  9119 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:59.448639  9118 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:59.417034  9191 rpcz_store.cc:276] 0709 13:04:56.881109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.881251 (+   142us) service_pool.cc:224] Handling call
0709 13:04:56.950193 (+ 68942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.468171 (+1517978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.382621 (+914450us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400143554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:59.382784 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3488546560}
I20260709 13:04:59.418964  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.422501  9162 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:04:59.441136  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.384003 12815 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.441675  9263 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:59.443931  9189 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:59.452952  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.453570  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.454219  9271 raft_consensus.cc:2368] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8.
I20260709 13:04:59.454361  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:59.455473  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 222519us
I20260709 13:04:59.457379  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.411226 12820 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:59.460000  9826 leader_election.cc:336] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.000s (SENT)
I20260709 13:04:59.460424  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:59.461199  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 187917us
W20260709 13:04:59.462324  9087 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.105s (SENT)
I20260709 13:04:59.462474  9263 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:04:59.462829  9087 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:59.464881  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.483908  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:59.485615  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.487183  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.488252  9271 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:04:59.437700 12423 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:59.489275  9195 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:04:59.489748  9826 leader_election.cc:336] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.080s (SENT)
I20260709 13:04:59.495003  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:59.497200  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.497898  9115 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:59.451467 12823 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:59.491575  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.445446 12792 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:59.492854  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:59.491649  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.501708  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:04:59.503536  9087 leader_election.cc:336] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.171s (SENT)
I20260709 13:04:59.508252  9087 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:59.503633  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.504949  9272 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:04:59.505259  9191 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:04:59.510113  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.460161 12593 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:59.497100  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:59.510859  9272 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:04:59.511924  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.512432  9118 raft_consensus.cc:2410] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:04:59.511868  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 247477us
W20260709 13:04:59.560510  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56994.9us
I20260709 13:04:59.513123  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2025) took 1087 ms. Trace:
I20260709 13:04:59.486795 12769 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:04:59.557787  9826 leader_election.cc:336] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.970s (SENT)
I20260709 13:04:59.496374 12815 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:59.552202  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.563125  9122 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:59.564152  9164 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:59.445086 12794 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.451349 12822 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
W20260709 13:04:59.561285  9087 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.179s (SENT)
I20260709 13:04:59.567521  9271 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:59.570670  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.570921  9087 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:59.499507 12807 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:04:59.500123 12567 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:04:59.571460  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.513032 12593 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.573319  9122 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:04:59.578114  9271 raft_consensus.cc:2410] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:04:59.561447  9196 rpcz_store.cc:276] 0709 13:04:58.425964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.435837 (+  9873us) service_pool.cc:224] Handling call
0709 13:04:58.458997 (+ 23160us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:04:58.474628 (+ 15631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:58.474670 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:04:58.490403 (+ 15733us) log.cc:844] Serialized 52 byte log entry
0709 13:04:58.825780 (+335377us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800444280. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:04:58.876266 (+ 50486us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:59.286856 (+410590us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.504420 (+217564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.504493 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.504532 (+    39us) raft_consensus.cc:1685] finished
0709 13:04:59.512893 (+  8361us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.513068 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":175999,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":69463,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":13527,"spinlock_wait_cycles":29764480,"thread_start_us":630482,"threads_started":3,"wal-append.queue_time_us":832992}
I20260709 13:04:59.559911 12755 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:59.586194  9263 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
W20260709 13:04:59.583732  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 86733.5us
I20260709 13:04:59.588303  9272 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:59.589294  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.588485  9161 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.589834  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.569020 12834 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.589320  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.592271  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.513693 12819 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:04:59.593680  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:59.594408  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.596361  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.587270 12593 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.601127  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.612272 12846 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:59.587512 12847 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:04:59.614567  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.615352  9122 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:59.615553 12776 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.617856 12815 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:04:59.618374 12815 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:59.619433 12815 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.620430  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.620697 12815 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:04:59.621363  9265 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:59.622192  9265 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c089688bfc69403095596a002ccbf890"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:04:59.622718  9265 raft_consensus.cc:606] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.623138  9265 raft_consensus.cc:614] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:59.624432  9190 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:59.625761  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.588675 12797 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:59.625819  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.630574  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.631459  9271 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:04:59.516316 12831 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.634351  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.594415 12833 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:04:59.633141  9272 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:59.635406  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 755) took 1421 ms. Trace:
I20260709 13:04:59.637729  9266 rpcz_store.cc:276] 0709 13:04:58.213893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.215906 (+  2013us) service_pool.cc:224] Handling call
0709 13:04:59.634178 (+1418272us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:04:59.635334 (+  1156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7944960}
I20260709 13:04:59.638783  9263 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:04:59.639029  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 77774.1us
I20260709 13:04:59.640214  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.640673  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.641482  9118 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:04:59.595220 12844 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:04:59.643854  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.642534  9263 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:04:59.645491  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:04:59.645731 12841 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:04:59.646080  9119 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:04:59.646665  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:59.647274 12838 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:04:59.648011  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.649070  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:59.649991  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.648619  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.650563  9263 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:04:59.650760  9117 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:59.649889  9272 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:04:59.651768  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.650450  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.628510 12797 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:59.653810  9263 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:04:59.654306  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:59.656140  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.653844  9271 raft_consensus.cc:2376] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9. Re-sending same reply.
I20260709 13:04:59.657491  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:59.634560 12821 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:04:59.664170  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.660514  9189 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:04:59.659003  9164 consensus_peers.cc:597] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c199cae5ff14a478726c8a8b491f8f7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.654275  9272 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:04:59.664459  9266 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:04:59.665769  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.667135  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.638700 12725 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.597178 12834 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:59.669158  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.676798  9118 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:04:59.669373  9828 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.677469  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.672785  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.672931 12812 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:04:59.673741  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.680609  9195 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:04:59.680522  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.681293  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.659795 12833 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:59.678653  9266 raft_consensus.cc:2376] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6. Re-sending same reply.
I20260709 13:04:59.675310  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.670102  9119 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:04:59.657940 12797 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:59.687045  9271 raft_consensus.cc:2376] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7. Re-sending same reply.
I20260709 13:04:59.687642  9272 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:04:59.680150  9117 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:59.688414  9272 raft_consensus.cc:606] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:59.678138  9263 raft_consensus.cc:2376] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4. Re-sending same reply.
I20260709 13:04:59.682258 12847 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:04:59.691949  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.675474 12811 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:59.693595  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.693827  9189 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:04:59.696348  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:59.686849 12804 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 1.188s	user 0.007s	sys 0.011s
I20260709 13:04:59.703019  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.689306 12776 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.720131  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.685278 12769 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:04:59.693503 12839 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:04:59.729697  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.678900 12725 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:59.739106  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.740073  9263 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:59.729208 12769 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.751017  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.751932  9263 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:59.704727 12474 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.762547  9263 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:56116
I20260709 13:04:59.763172  9263 raft_consensus.cc:606] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:04:59.763527  9263 raft_consensus.cc:614] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:04:59.764459  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:04:59.765610  9272 raft_consensus.cc:2368] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8.
I20260709 13:04:59.768380  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2179) took 2594 ms. Trace:
I20260709 13:04:59.768841  9116 rpcz_store.cc:276] 0709 13:04:57.174434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.182943 (+  8509us) service_pool.cc:224] Handling call
0709 13:04:57.236225 (+ 53282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:04:57.400227 (+164002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:04:57.400302 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:04:57.406433 (+  6131us) log.cc:844] Serialized 26 byte log entry
0709 13:04:57.902428 (+495995us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:04:57.902485 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:57.902578 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:04:59.673812 (+1771234us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:04:59.763881 (+ 90069us) raft_consensus.cc:1685] finished
0709 13:04:59.763903 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.768312 (+  4409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9139,"raft.queue_time_us":29178,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":24975,"spinlock_wait_cycles":13019392,"thread_start_us":601866,"threads_started":2,"wal-append.queue_time_us":1037710}
I20260709 13:04:59.732887 12776 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.773860 12474 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.776276  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1036) took 1129 ms. Trace:
I20260709 13:04:59.776646  9861 rpcz_store.cc:276] 0709 13:04:58.646777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.651615 (+  4838us) service_pool.cc:224] Handling call
0709 13:04:58.658013 (+  6398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.379810 (+721797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.776054 (+396244us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54004a3dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:59.776220 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1564906880}
I20260709 13:04:59.779873 12793 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:04:59.780300 12793 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:04:59.781148 12793 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.782428 12793 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:04:59.783907  9828 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:59.788302  9270 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:04:59.789795  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:04:59.790216 12853 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:04:59.765689 12833 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.791970 12820 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:59.801935  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.812604 12834 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.814069 12776 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.843078  9263 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:04:59.814503 12593 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:59.820772  9018 catalog_manager.cc:5697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:59.827752  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:04:59.828688  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.832237  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.835258  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:04:59.833622  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.831029 12723 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.846673  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.852104  9270 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:04:59.846429  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:04:59.852581  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 154830us
I20260709 13:04:59.852128  9269 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:04:59.853956  9161 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:59.855993 12167 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.003s
I20260709 13:04:59.856446 12167 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.857649  9123 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
W20260709 13:04:59.860814 12855 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.071s	user 0.002s	sys 0.000s
I20260709 13:04:59.852836  9263 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:59.865159  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.831504 12794 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.868100  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.869599 12793 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:04:59.875759 12593 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.876516 12725 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:04:59.865494 12723 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.878427  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2028) took 1362 ms. Trace:
I20260709 13:04:59.878978  9190 rpcz_store.cc:276] 0709 13:04:58.515966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.516110 (+   144us) service_pool.cc:224] Handling call
0709 13:04:59.878225 (+1362115us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540010b654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:04:59.878370 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2509128064}
I20260709 13:04:59.852222 12820 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:04:59.883273 12423 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:04:59.886106 12593 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.890193 12593 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:59.883491 12638 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:04:59.888124 12751 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:04:59.896885 12751 raft_consensus.cc:3055] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:04:59.897315 12751 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:04:59.897836  9087 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.541s (SENT)
W20260709 13:04:59.898619  9087 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.542s (SENT)
I20260709 13:04:59.899107  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:59.881544  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:04:59.904160  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:59.904861  9861 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:04:59.905993  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:59.906627  9862 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:04:59.907290 12723 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:59.898195 12751 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:04:59.908577  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:04:59.908689  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:04:59.915552  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 47563.2us
I20260709 13:04:59.907575  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:04:59.910749  9825 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:04:59.911605  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.909567  9857 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: []
I20260709 13:04:59.916561  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:04:59.918442  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:59.909590  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 617) took 1389 ms. Trace:
I20260709 13:04:59.920436  9121 rpcz_store.cc:276] 0709 13:04:58.520367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.521121 (+   754us) service_pool.cc:224] Handling call
0709 13:04:58.521461 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:58.649483 (+128022us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:04:58.850346 (+200863us) log.cc:844] Serialized 14 byte log entry
0709 13:04:59.451701 (+601355us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:04:59.451776 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:04:59.451826 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.909366 (+457540us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54002f6cd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:04:59.909413 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:04:59.909537 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12388,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":625,"raft.queue_time_us":49828,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":78,"spinlock_wait_cycles":293993728,"thread_start_us":784099,"threads_started":3,"wal-append.queue_time_us":564940}
I20260709 13:04:59.916466  9858 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:04:59.922910  9087 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.725s (SENT)
I20260709 13:04:59.926529  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:59.884264 12858 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:04:59.939046 12423 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:04:59.941470  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59418.8us
I20260709 13:04:59.940083 12423 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:04:59.942608  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.915161 12747 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:04:59.944068 12844 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.944159 12733 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:04:59.944679 12733 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:59.944617 12844 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.945250 12733 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.921294 12609 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:04:59.915560 12792 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:04:59.958362  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.934628 12723 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:04:59.934963 12794 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:59.950125  9826 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.213s (SENT)
I20260709 13:04:59.967329  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:04:59.967552  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.970666  9826 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:04:59.970839  9115 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:04:59.935946 12776 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:59.971277  9114 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.971736  9198 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:04:59.956079  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.972294  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.944160  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.962270  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:04:59.972875  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.977507  9190 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:59.977095  9119 raft_consensus.cc:2410] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:04:59.978291  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:04:59.956874  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:59.982329 12747 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:59.982841 12747 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.983501 12844 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:04:59.984480 12854 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:04:59.985087 12747 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:59.986274 12797 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.987453 12805 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:59.988032  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:04:59.990331 12857 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.199s	user 0.000s	sys 0.002s
I20260709 13:04:59.992990  9114 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:04:59.993475  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2036) took 1073 ms. Trace:
I20260709 13:04:59.994181 12847 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:04:59.994865  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:04:59.941716 12851 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:04:59.993870  9192 rpcz_store.cc:276] 0709 13:04:58.919854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.920084 (+   230us) service_pool.cc:224] Handling call
0709 13:04:59.045839 (+125755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:04:59.232755 (+186916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:04:59.993217 (+760462us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b540001dc20. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f765665bdc0 00007f76566e7a02 00007f76566e3dd6 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:04:59.993415 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":524302720}
I20260709 13:04:59.942524 12755 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:04:59.960989 12751 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:04:59.981278  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:04:59.979772 12769 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:04:59.940938 12756 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:00.004915  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 104030us
W20260709 13:05:00.005162  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 55097.7us
I20260709 13:04:59.944100 12593 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:00.024015  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63748.5us
I20260709 13:04:59.980541 12167 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:04:59.937775 12819 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:05:00.025807  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:00.031180  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.021591 12567 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:00.000535 12798 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:00.003062 12755 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.032078  9115 raft_consensus.cc:2410] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:04:59.960718 12823 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:05:00.001940 12851 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.023051 12807 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:00.069006 12805 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:00.069269 12807 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.059514 12733 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.069634 12805 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.034407  9238 leader_election.cc:400] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10.
I20260709 13:05:00.041179 12769 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:00.041623 12755 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:04:59.980060 12792 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.044601 12819 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:00.050853 12797 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.055351 12583 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:00.056313  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:00.056910  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:04:59.979895 12794 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:00.037777 12840 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.535s	user 0.003s	sys 0.000s
I20260709 13:05:00.058192  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.060724  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.062350 12823 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:00.066241 12798 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.067448 12780 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:00.057492 12850 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:00.003207 12820 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.069185 12851 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.069880 12807 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.043113  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:00.068126  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.071506 12733 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.133121  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:00.086056  9861 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:05:00.141144  9238 leader_election.cc:403] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:00.142648 12583 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.143091  9268 raft_consensus.cc:2410] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:00.141201 12823 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:00.079282 12755 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.122414  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.072052  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:00.144158  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 72187.8us
I20260709 13:05:00.144259 12723 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:00.144894 12723 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.145044 12755 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.145270 12723 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:00.143217  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 108957us
I20260709 13:05:00.146188  9238 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.143245 12583 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.142683 12751 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10.
I20260709 13:05:00.148007 12864 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:00.130980 12848 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.149354  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:00.120913 12849 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:05:00.154651  9192 raft_consensus.cc:2376] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6. Re-sending same reply.
I20260709 13:05:00.155277 12864 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:00.155718 12864 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.156164 12864 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.156672 12794 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.156950 12792 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.090646  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.141801 12819 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:05:00.157881  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 67353.6us
I20260709 13:05:00.158456 12751 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:00.158573  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.158981 12751 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.159436 12747 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:00.159436 12751 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.159880 12747 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.160281 12747 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:00.123873  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:00.165336 12823 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.165737 12819 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.167491 12819 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:00.167302 12747 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:00.166610 12823 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:00.169538  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.149295 12848 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.170499 12794 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.171015 12794 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.171705 12848 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.173303  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:00.075131  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:00.174468  9854 raft_consensus.cc:2393] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:00.129539  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.175678 12798 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:00.180413  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:00.180732  9018 catalog_manager.cc:5697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 4 to 5, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 5 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.184296 12583 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.183452  9018 catalog_manager.cc:5697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 3, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:00.173281 12794 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:00.141947 12850 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.071571 12805 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.202356  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.203418  9192 raft_consensus.cc:2410] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:05:00.204131  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.205606 12780 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:00.206516 12797 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:00.147372  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.207691 12805 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:00.208623  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 61312.3us
I20260709 13:05:00.209332  9197 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:00.192703 12751 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:00.210983  9197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2030) took 1644 ms (client timeout 1707 ms). Trace:
I20260709 13:05:00.197918  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.212092  9267 raft_consensus.cc:2393] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:00.186869 12864 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.210189  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:00.213276 12805 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:00.206132 12850 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.213717 12805 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.214372 12805 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:00.202414  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:00.211438  9197 rpcz_store.cc:269] 0709 13:04:58.566453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:58.566721 (+   268us) service_pool.cc:224] Handling call
0709 13:05:00.210827 (+1644106us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b540010c554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:00.210929 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3622369024}
I20260709 13:05:00.217190  9194 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:00.217583  9122 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:00.169576 12747 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:00.206763  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.214448 12864 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.220564 12794 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:00.221199 12864 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:00.221262 12794 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.224756 12794 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.225165 12805 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:00.228858  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.232025  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:00.212245 12751 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:00.233477 12794 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:00.233748  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.234715  9194 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:00.233840  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.235703  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:00.235901  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 76175.8us
I20260709 13:05:00.239323  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:00.239975  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:00.241389 12819 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:00.241915  9861 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:00.242079 12819 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.242228  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.242515 12819 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.240590 12769 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.243165 12769 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.243589 12751 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:00.244118 12751 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:00.245306 12769 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:00.244973 12751 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.246076  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.246506  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:00.247160  9123 raft_consensus.cc:2393] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:00.234110 12794 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.249320  9861 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:00.249573 12794 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:00.252149  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:00.252841  9123 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.253952 12792 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:00.254158 12583 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.249362 12819 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.257750  9861 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:00.258292 12819 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:00.259016 12848 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:00.260393  9190 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:00.260661  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:00.261320  9120 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.262436  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:00.263187  9265 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:00.268605  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.270159  9265 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:05:00.271327  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:00.271999  9267 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.277794  9120 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:00.280182  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.281323 12878 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:00.282244 12878 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:00.283085  9018 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:00.284742  9018 catalog_manager.cc:5697] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:00.285945 12794 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.286096  9018 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 3 to 5, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 5 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:05:00.294231  9019 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36610 (request call id 325) took 24360 ms (client timeout 15000 ms). Trace:
I20260709 13:05:00.283289 12878 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.298120 12723 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.278851 12792 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:00.301924  9861 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:05:00.241097  9854 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:05:00.305716  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 73179.7us
I20260709 13:05:00.307614 12797 consensus_queue.cc:1243] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:05:00.309929  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 76298.3us
I20260709 13:05:00.310801  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:00.308513  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 102007us
I20260709 13:05:00.312242  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.252426  9122 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:00.333388  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:00.334003  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.336589  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.339260  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.342123  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:00.344148  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:00.354875  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.354774  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:00.356204  9238 connection.cc:256] RPC call timeout handler was delayed by -2.2482818590000306s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 13:05:00.356580  9238 connection.cc:256] RPC call timeout handler was delayed by -2.1222818590000543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
W20260709 13:05:00.356873  9238 connection.cc:256] RPC call timeout handler was delayed by -2.1222818590000543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
W20260709 13:05:00.357170  9238 connection.cc:256] RPC call timeout handler was delayed by -2.0583818590000647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
W20260709 13:05:00.357465  9238 connection.cc:256] RPC call timeout handler was delayed by -1.9584818590000168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
W20260709 13:05:00.357749  9238 connection.cc:256] RPC call timeout handler was delayed by -1.8990818590000345s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 13:05:00.358006  9238 connection.cc:256] RPC call timeout handler was delayed by -1.8873818590000155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1968s for a response
W20260709 13:05:00.358256  9238 connection.cc:256] RPC call timeout handler was delayed by -1.8819818590000068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
W20260709 13:05:00.358505  9238 connection.cc:256] RPC call timeout handler was delayed by -1.8675818589999835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
W20260709 13:05:00.358768  9238 connection.cc:256] RPC call timeout handler was delayed by -1.8675818589999835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
W20260709 13:05:00.359019  9238 connection.cc:256] RPC call timeout handler was delayed by -1.8567818590000797s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2002s for a response
W20260709 13:05:00.359331  9238 connection.cc:256] RPC call timeout handler was delayed by -1.851381859000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
I20260709 13:05:00.360085  9828 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:00.360533  9123 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:00.361225  9828 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:00.362250  9238 connection.cc:256] RPC call timeout handler was delayed by -1.8162818590000143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 13:05:00.362758  9238 connection.cc:256] RPC call timeout handler was delayed by -1.7829818590000741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
W20260709 13:05:00.363101  9238 connection.cc:256] RPC call timeout handler was delayed by -1.7541818590000275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
W20260709 13:05:00.363729  9238 connection.cc:256] RPC call timeout handler was delayed by -1.6344818590000614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
I20260709 13:05:00.354774  9018 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 5 to 6, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 6 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:00.381599  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 69714.7us
I20260709 13:05:00.380896  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:00.372028  9119 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:00.382678  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:00.384644  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.385450  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:00.386001  9267 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:00.386284  9194 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:00.385492  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.387936  9190 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:00.390627  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.391180  9114 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:05:00.392571  9267 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:05:00.393146  9267 pending_rounds.cc:85] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 9
I20260709 13:05:00.393719  9267 pending_rounds.cc:107] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:05:00.395407  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.394189  9267 raft_consensus.cc:2889] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:00.396183  9118 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:00.308573 12850 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:00.307607 12780 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:00.399789  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:00.400024  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.312764 12805 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.385062  9264 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:05:00.406100  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.406507  9238 leader_election.cc:400] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10.
I20260709 13:05:00.407367  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.360607 12792 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:00.408857  9858 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:00.404472  9854 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:00.410874  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.411731  9114 raft_consensus.cc:2410] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:05:00.411149  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.383313 12474 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:00.413040  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 71041.2us
I20260709 13:05:00.416244  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.409874  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.401659  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:00.364331 12858 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:00.413892  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.419615  9856 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:05:00.419847  9825 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.390419 12769 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.383383 12849 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:56644 (request call id 18) took 2879 ms. Trace:
I20260709 13:05:00.384501 12820 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:00.391688 12823 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:00.448401  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.450176  9858 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:00.291479 12851 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.418502  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:00.308316 12583 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 13:05:00.422787  9194 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:00.413983  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.423521  9018 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 3 to 10, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 10 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:00.427465 12726 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:00.415232 12831 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:00.294754  9019 rpcz_store.cc:269] 0709 13:04:35.973031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:35.973325 (+   294us) service_pool.cc:224] Handling call
0709 13:05:00.293984 (+24320659us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:54.527216 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:54.536847 (+  9631us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:54.536942 (+    95us) write_op.cc:435] Acquired schema lock
0709 13:04:54.536987 (+    45us) tablet.cc:662] Decoding operations
0709 13:04:54.539211 (+  2224us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:54.539325 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:54.539376 (+    51us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:04:54.541374 (+  1998us) tablet.cc:701] Row locks acquired
0709 13:04:54.541443 (+    69us) write_op.cc:260] PREPARE: finished
0709 13:04:54.541698 (+   255us) write_op.cc:270] Start()
0709 13:04:54.541944 (+   246us) write_op.cc:276] Timestamp: P: 1783602294541609 usec, L: 0
0709 13:04:54.541972 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:04:54.543392 (+  1420us) log.cc:844] Serialized 5562 byte log entry
0709 13:04:55.005718 (+462326us) op_driver.cc:464] REPLICATION: finished
0709 13:04:55.458003 (+452285us) write_op.cc:301] APPLY: starting
0709 13:04:55.458139 (+   136us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:55.548924 (+ 90785us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:55.584787 (+ 35863us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:55.680831 (+ 96044us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:55.681238 (+   407us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:04:55.681290 (+    52us) write_op.cc:312] APPLY: finished
0709 13:04:55.681727 (+   437us) log.cc:844] Serialized 101 byte log entry
0709 13:04:55.681949 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:55.682106 (+   157us) write_op.cc:454] Released schema lock
0709 13:04:55.682175 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:55.682623 (+   448us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:55.682656 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3988928,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":444082,"apply.run_cpu_time_us":6614,"apply.run_wall_time_us":232486,"num_ops":14,"prepare.queue_time_us":416227,"prepare.run_cpu_time_us":49664,"prepare.run_wall_time_us":275025,"raft.queue_time_us":93662,"raft.run_cpu_time_us":63042,"raft.run_wall_time_us":297849,"replication_time_us":463587,"thread_start_us":771332,"threads_started":4,"wal-append.queue_time_us":261638,"wal-append.run_cpu_time_us":18353,"wal-append.run_wall_time_us":1874698}]]}
I20260709 13:05:00.428299 12776 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:00.431516 12780 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.423347 12793 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:00.423715 12794 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.478399 12794 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.478884 12794 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.473521 12755 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:05:00.481442 12831 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:00.481341 12807 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.482496 12831 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.434419 12769 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.417305 12798 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.424041 12792 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:00.423890 12567 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:00.428576 12847 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:00.418795 12474 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:05:00.453474  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.429798  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:00.476615  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.443707  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.439069  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.404183 12850 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:00.445185  9087 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.461812  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:00.488366  9267 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.429477 12878 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:00.494031  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45718.7us
I20260709 13:05:00.499446  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:00.455549  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:00.463891  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:00.452521 12805 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:05:00.425786  9123 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index -1 to 6, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:05:00.494760  9854 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:00.463027  9825 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.491303  9861 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
W20260709 13:05:00.515363  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52483.4us
I20260709 13:05:00.447559 12823 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:00.515163  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:00.491431  9855 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:05:00.544236  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 44902.8us
I20260709 13:05:00.543996  9018 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:00.487423  9122 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:05:00.553574  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.548166  9856 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:00.548823  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:00.533448  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 114695us
I20260709 13:05:00.485674 12474 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:00.485177 12751 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:00.492558  9089 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.517006  9189 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:00.559456  9162 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.561604  9267 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:00.590159  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.452735 12618 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.429064 12858 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.473433 12822 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:05:00.494827 12869 ts_tablet_manager.cc:933] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:00.555258 12823 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:00.569190 12776 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:00.483852 12793 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:00.580902 12851 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.488492 12723 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.510524 12567 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:00.585186 12798 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.485488 12847 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.451184 12167 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.104s
W20260709 13:05:00.591789  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 81513.6us
W20260709 13:05:00.592686  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43960.5us
I20260709 13:05:00.495344 12850 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.483682 12807 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.601467  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.573554  9018 catalog_manager.cc:5697] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 4 to 7, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 7 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:00.551265 12747 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:00.556522 12848 raft_consensus.cc:2764] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:05:00.542328 12885 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:00.645704 12893 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:00.519464 12878 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:00.646243 12893 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.646509 12878 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:00.646085  9018 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:00.534309 12820 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:00.614176 12890 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:05:00.647712  9018 catalog_manager.cc:5697] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 6, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 6 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:05:00.648200 12820 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.608076 12618 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.649791 12747 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:00.650125  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48768.8us
I20260709 13:05:00.617178 12888 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:00.650810 12888 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.649758  9018 catalog_manager.cc:5697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 2 to 6, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 6 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:00.651510 12618 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.616413 12474 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.619304  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:00.588508 12751 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:00.652371 12751 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.653095 12609 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:00.653442  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:00.653580 12474 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:00.494825 12755 consensus_queue.cc:1059] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:05:00.654848 12755 raft_consensus.cc:3055] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:00.655052 12797 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:00.655218 12755 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:00.593114 12877 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:00.656174 12755 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.656525 12877 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:00.650866  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.657065 12877 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:00.657256  9856 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:00.636484  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:00.494699 12756 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:00.485198 12792 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:00.644196  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 199081us
I20260709 13:05:00.646656 12893 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:00.651273 12888 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:00.686859 12423 consensus_queue.cc:1175] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "92919b70789041c8a0de0b3525154fd1" responder_term: 9 status { last_received { term: 9 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 9 index: 9 } }
I20260709 13:05:00.717999 12893 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.604115 12864 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:00.719501  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.660038 12747 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:00.660858  9855 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:00.720779  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.721058 12807 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:00.721493 12798 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:00.721735 12891 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:00.722096 12895 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.720367 12893 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.665397  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:00.727865  9194 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:00.700382  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:00.636173 12567 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.636260 12847 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:00.629191 12833 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:05:00.634533 12887 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:00.627841 12846 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:05:00.690336 12866 consensus_queue.cc:1059] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:05:00.629434 12886 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:00.685853 12756 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.706210  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.708806  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.709227  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.709519  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.775584  9198 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:00.775605  9192 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:00.707679  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:00.711506  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:00.713949 12883 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:05:00.716816 12858 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.716442 12776 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.781134  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:00.781770  9193 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:00.631434  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 138958us
I20260709 13:05:00.784967  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.725323  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:00.786266  9855 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:00.786206  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.626559  9854 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:00.738920  9194 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
W20260709 13:05:00.740527  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 46644.5us
I20260709 13:05:00.741587  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:00.789986  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.727063  9018 catalog_manager.cc:5697] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 1 to 3, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 3 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:00.750747  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.722559 12798 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:00.705641 12474 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:00.767910  9263 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:00.768286  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.792776  9828 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:00.705452 12797 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:00.776664  9195 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:00.777371  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.780295 12567 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:00.777453 12756 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:00.806986  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.764056 12807 raft_consensus.cc:2764] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:05:00.706559 12878 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.797874  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:00.797787  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.812058  9123 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:00.798943  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.808053  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.812637  9193 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:00.809664  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.778298 12869 tablet_copy_client.cc:323] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.66:34417
I20260709 13:05:00.737112 12834 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:05:00.814838  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.815217  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:00.821762  9859 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:00.817070  9123 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:00.817286  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:00.820240  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:00.829644  9854 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:00.852730  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.853134  9120 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:00.853669  9189 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:00.856323  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:00.859097  9193 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:00.792181 12798 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:00.798502 12805 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:00.863536  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.742601 12793 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.798687 12167 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:00.870054  9197 raft_consensus.cc:2376] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10. Re-sending same reply.
I20260709 13:05:00.427945 12849 rpcz_store.cc:276] 0709 13:04:57.474796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.474908 (+   112us) service_pool.cc:224] Handling call
0709 13:04:58.522951 (+1048043us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b4c00502180. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f76549b0a20 00007f76549c1a29 00007f7654987644 00007f76549aef2d 00007f76549aeb93 00007f765663d1af 00007f765663cf3e 00007f76566c2888 00007f76566df75c 00007f76566e076a 00007f76566df91b
0709 13:04:58.525499 (+  2548us) log.cc:844] Serialized 499 byte log entry
0709 13:04:59.249127 (+723628us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b58007217d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664e090 00007f765663e43c 00007f7656644d53 00007f7656644cd9 00007f7656644c69 00007f7656644c31 00007f7656643dfd 000000000038f354 000000000038f199 00007f76535f7475 00007f76535f97c1
0709 13:04:59.694497 (+445370us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b580031e8d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664e090 00007f765663e43c 00007f7656644d53 00007f7656644cd9 00007f7656644c69 00007f7656644c31 00007f7656643dfd 000000000038f354 000000000038f199 00007f76535f7475 00007f76535f97c1
0709 13:05:00.354280 (+659783us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44859,"raft.queue_time_us":22001,"raft.run_cpu_time_us":9979,"raft.run_wall_time_us":1640238,"spinlock_wait_cycles":2597896832,"thread_start_us":95805,"threads_started":1,"wal-append.queue_time_us":129177,"wal-append.run_cpu_time_us":1951,"wal-append.run_wall_time_us":1099705}
I20260709 13:05:00.722640 12895 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.870597  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:00.871402  9192 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:00.866667  9123 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:00.867959  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:00.872745  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 78051.2us
I20260709 13:05:00.814392 12844 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:00.878623  9854 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:00.873529  9238 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.877122  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:00.877430  9268 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:00.883734  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.857664 12875 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:00.888217  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:00.892948 12807 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:00.892973 12167 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:00.894840 12474 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:00.795497 12823 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.808583 12866 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:05:00.927632  9117 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:00.938494 12877 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.939970  9269 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:00.941883 12877 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:00.878291 12805 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:00.897136  9197 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:00.896291  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:00.901648  9123 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:00.896036 12904 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:00.910033 12797 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:00.918941 12723 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:00.920354 12902 raft_consensus.cc:3055] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:05:00.925931 12896 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:00.926756 12567 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.895795  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:00.988679  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 93006.3us
W20260709 13:05:00.954551  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 203932us
I20260709 13:05:00.955516  9859 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:00.939388 12751 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:00.998469  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.909426 12820 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:00.906013 12864 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:00.993141  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.981237 12805 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:01.001338  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 105343us
I20260709 13:05:01.005368  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:00.999512 12820 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:01.008526  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 107701us
I20260709 13:05:01.009147  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:00.987895 12723 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:01.016633  9264 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:00.999869 12894 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:01.017889  9087 leader_election.cc:400] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8.
I20260709 13:05:01.001348 12618 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:01.021018  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:00.997645 12904 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:01.030295  9195 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:00.988269 12902 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:01.022501  9087 leader_election.cc:403] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:05:01.023952  9197 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:05:00.957190 12798 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:01.032140  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.011935 12820 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:01.015005 12893 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:01.033156 12167 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.010865 12844 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:01.010525 12807 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:01.004561 12567 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.050043  9207 tablet_copy_service.cc:140] P 94504d29c6744f38bb06943479df9466: Received BeginTabletCopySession request for tablet 1c199cae5ff14a478726c8a8b491f8f7 from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:56646)
I20260709 13:05:01.051030 12769 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:01.051424 12903 log.cc:927] Time spent T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:01.052884 12852 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:01.054589 12846 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:01.047431 12859 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.057928  9089 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:01.065680 12891 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8.
I20260709 13:05:01.065538 12725 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:01.065796 12583 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:01.066792 12895 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.066941 12723 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.068069 12904 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.069239 12902 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.070420 12751 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:01.073158  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.038161 12860 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.056151 12908 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:01.102787  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 85372.1us
W20260709 13:05:01.106042  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 232592us
I20260709 13:05:01.047343 12851 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:01.076138 12899 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:05:01.113034  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:01.119820  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 61980.1us
I20260709 13:05:01.116242 12846 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.130806 12733 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:05:01.168174  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.026180 12474 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.114504 12823 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.136617  9856 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:05:01.169098 12919 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:01.113061  9207 tablet_copy_service.cc:161] P 94504d29c6744f38bb06943479df9466: Beginning new tablet copy session on tablet 1c199cae5ff14a478726c8a8b491f8f7 from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:56646: session id = 92919b70789041c8a0de0b3525154fd1-1c199cae5ff14a478726c8a8b491f8f7
W20260709 13:05:01.137362  9826 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.503s (SENT)
I20260709 13:05:01.147454 12386 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:05:01.148473 12794 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:01.047916 12887 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:05:01.105365 12878 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.105726 12807 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:01.117774 12618 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:01.186780 12618 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.188000  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:01.188519 12567 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:01.164649  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:01.168371  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:01.155519 12905 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:01.136149 12859 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.212752  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:01.213668  9117 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:01.175421  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:01.185464 12807 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:01.216698  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 52108.8us
I20260709 13:05:01.186828 12820 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:01.175822  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 62875.9us
W20260709 13:05:01.189595 12913 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:05:01.217867  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:01.224792  9117 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:01.192476  9855 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:05:01.222232  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 84893.1us
W20260709 13:05:01.226127  9826 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.552s (SENT) [suppressed 70 similar messages]
I20260709 13:05:01.229413  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:01.229154  9087 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:01.232213  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:01.157981 12805 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.116472 12852 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.214094 12831 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:01.189694  9207 tablet_copy_source_session.cc:215] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:01.193002 12902 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:01.238935  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:01.212364 12905 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.243052  9859 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:05:01.245090  9087 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.223425 12723 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:01.214610 12894 raft_consensus.cc:697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:01.228375 12887 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:01.227008 12878 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:01.253592  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.253871  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.254509  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.268602  9266 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:01.269199  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.246184 12638 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:01.233067 12794 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:01.237496 12924 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:01.248981 12893 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:01.261694  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.175334 12877 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:05:01.257620  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.278908  9267 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:01.279101  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.257486  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.265630  9122 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:01.216843 12769 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:05:01.175963 12851 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:01.269752  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 43633.6us
I20260709 13:05:01.257236  9270 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:01.271437  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.165465 12860 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.273727  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.273200 12638 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:01.280836  9265 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:01.252265 12905 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:01.246487 12858 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:05:01.293496  9087 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.293012  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:01.295368  9826 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.648s (SENT)
I20260709 13:05:01.295853  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:01.298221  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.298494  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.299414  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.299902  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.300294  9270 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:01.273838 12794 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.274660 12924 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.301502  9117 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:01.301573  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.302177  9267 raft_consensus.cc:2376] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8. Re-sending same reply.
I20260709 13:05:01.278904 12891 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:01.304103  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.304200  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.273669 12893 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:01.304634 12925 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:01.307952  9281 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:05:01.314010  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.320645  9123 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:01.306306  9265 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:01.315438  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.318926  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.319564  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.300017 12167 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:01.283479 12886 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:01.324262  9272 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:01.297080 12638 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:01.329959  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:01.335445  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:01.316896 12847 consensus_meta.cc:287] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.003s
I20260709 13:05:01.331380  9271 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:01.317313 12893 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:01.376544 12805 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.329044 12894 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:01.378327 12917 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:05:01.306265 12794 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:01.330811  9281 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 685fa3ec557f4f3db02ec6ccce1d6f3e from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-685fa3ec557f4f3db02ec6ccce1d6f3e
I20260709 13:05:01.384316  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:01.323024 12888 consensus_meta.cc:287] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.003s
I20260709 13:05:01.371272 12850 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:01.316648 12567 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:05:01.372581 12929 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:05:01.372928  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.397186  9267 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:01.373620  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.374424 12847 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.375181 12820 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:01.313058 12924 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:01.295014 12756 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.388041  9271 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:01.393692  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:01.394047  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.367059  9196 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:01.330583  9270 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:01.396088  9191 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:01.398897  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.399509  9265 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:01.408504  9267 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:01.408460  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.409154  9270 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:01.410310  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:01.410619  9265 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:01.410625  9117 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:01.412943  9122 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:01.413398  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.415046  9861 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:01.419138 12756 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.419912  9120 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:01.394649 12888 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.421526 12859 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.422034 12888 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:01.422829  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:01.444379  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.462623 12847 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:01.477388  9826 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.112s (SENT)
W20260709 13:05:01.682276  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 204934us
I20260709 13:05:01.684001  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.512311 12852 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.416836 12877 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:05:01.521054  9267 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:01.544185  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:01.544888  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:01.687713  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:01.545166  9189 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:01.545662  9854 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:01.427526 12919 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
W20260709 13:05:01.532770 12870 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.067s	user 0.003s	sys 0.000s
I20260709 13:05:01.547912  9190 raft_consensus.cc:2410] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:05:01.554523 12638 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.564898  9192 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:01.568764  9271 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:01.571305 12756 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.538784  9281 tablet_copy_source_session.cc:215] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:05:01.591248  9191 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:01.539259 12908 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:01.614966  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.643575 12893 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.645316 12794 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:01.700273 12933 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:05:01.701339 12794 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:01.645710 12924 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.701953  9197 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:01.703101 12924 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.703204 12638 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.652302  9855 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:01.657471  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:01.658731  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:01.662132  9193 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:01.647549 12891 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:01.708523 12934 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 13:05:01.680756  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:01.684067  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.685642 12888 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:01.650557 12905 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:01.709977 12794 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:01.710510  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:01.709991 12888 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.711669 12905 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:01.708001  9114 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:01.461160 12878 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:01.686200  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.689638 12925 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:01.689877 12902 raft_consensus.cc:3055] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:01.693677  9857 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:01.693770  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:01.694401  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:01.688128 12820 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:01.462364 12851 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.708036 12893 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.714327  9856 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:05:01.718597  9266 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:01.724458 12820 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.715231  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.725703 12794 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:01.726698 12820 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:01.716336 12756 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.729002  9087 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.729791 12902 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:01.730856 12902 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.732223  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.732366  9087 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:01.736764  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:01.716300 12908 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:05:01.740733 12887 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:01.734292 12924 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:01.716073 12888 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:01.729739 12756 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:01.761746  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.761152  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:01.770613  9194 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
W20260709 13:05:01.747457  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.775672  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:01.776371  9193 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:01.734872 12925 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:01.777623  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:01.779037  9194 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:05:01.782140  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:01.736138 12893 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:01.782744  9194 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:01.782603  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:01.738587 12847 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:01.786113  9192 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:01.786897  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:01.746841 12902 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:01.790505  9856 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
W20260709 13:05:01.791503  9069 auto_rebalancer.cc:629] Failed to schedule move for tablet dfa5110f517a4ffb866953379f5d0c4d: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:05:01.792526  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:01.805217  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:01.806037  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:01.806633  9195 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:01.806650  9194 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:01.794059 12860 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:01.794191  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:01.815428  9192 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:01.817446 12751 consensus_queue.cc:1059] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:01.818620 12751 raft_consensus.cc:3055] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:01.819075 12751 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:01.797842 12869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c199cae5ff14a478726c8a8b491f8f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:01.740805 12851 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:05:01.798326  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:01.734784 12820 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:05:01.741271 12887 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:01.801393 12935 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.340s	user 0.018s	sys 0.025s
I20260709 13:05:01.810431  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:01.748677 12924 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:01.796342 12755 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:01.823318  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:01.850425  9856 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:01.825914  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:01.826180  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:01.851643  9860 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:05:01.826808  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:01.827389  9193 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:01.852792  9857 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:01.829924  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:01.832060  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:01.828708 12851 consensus_queue.cc:1243] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:01.840632 12887 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:01.829392  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:01.772238 12894 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:05:01.844660  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:01.866667  9123 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:01.846395  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:01.849411  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:01.867730  9114 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:01.868225  9119 raft_consensus.cc:2368] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9.
I20260709 13:05:01.828428 12847 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:01.865315  9122 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:01.836000  9195 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:01.851406 12924 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:01.873500  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 44150.9us
I20260709 13:05:01.836771 12908 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:05:01.857348 12751 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:01.887224  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 200439us
I20260709 13:05:01.856591 12904 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:01.955029  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:01.960259 12932 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:05:01.960747 12866 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:01.961581 12866 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:01.874241 12847 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.967533 12847 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:01.875491 12567 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:01.875700 12858 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:01.862795 12888 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:05:01.886251 12593 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:01.986349 12820 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.990805 12593 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:02.012176 12878 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:02.050194  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 175650us
W20260709 13:05:02.061932  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 185927us
I20260709 13:05:02.041172 12794 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:02.063081  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:02.065809  9862 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:02.066795  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:02.035797 12820 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:02.074378  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 119504us
I20260709 13:05:02.075192  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:02.077540  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:02.080418 12474 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:02.086647  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:02.091150  9238 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:02.069492 12860 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:02.094153  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:02.071161 12904 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:02.104748  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:02.085353 12878 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:02.076726 12851 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:02.125988  9238 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:02.107519  9860 pending_rounds.cc:85] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 8
I20260709 13:05:02.134166  9860 pending_rounds.cc:107] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:05:02.134425  9860 raft_consensus.cc:2889] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:02.089512 12906 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:02.136336  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:02.092617 12888 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:02.117743 12919 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:02.097241 12860 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.142779 12820 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:02.143054 12860 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:02.143209 12908 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:02.143491  9161 leader_election.cc:400] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9.
I20260709 13:05:02.144220  9161 leader_election.cc:403] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:05:02.145364  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:02.146317  9197 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
W20260709 13:05:02.146557  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54836.8us
I20260709 13:05:02.129588 12851 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.070019 12593 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.131251 12878 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:02.133457 12956 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:02.160333  9010 maintenance_manager.cc:419] P cdc5c504a10a4da1812e571d14ae3abd: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033383
I20260709 13:05:02.166118 12952 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:02.171867 12851 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:02.195801  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.178125 12888 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:02.181938 12906 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:02.203305  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 62388.5us
I20260709 13:05:02.149498 12908 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.181908 12685 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
W20260709 13:05:02.208952 12951 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:05:02.212322 12944 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:05:02.195472 12866 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:02.202316 12904 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:02.224206 12923 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.953s	user 0.002s	sys 0.011s
I20260709 13:05:02.224601 12956 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:02.229112  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2226) took 1002 ms. Trace:
I20260709 13:05:02.230942  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 1065) took 1414 ms. Trace:
I20260709 13:05:02.233628 12952 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9.
I20260709 13:05:02.196714 12919 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:02.236843 12963 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:02.238934  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:02.295277  9264 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:02.296217  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:02.296820  9264 raft_consensus.cc:2376] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9. Re-sending same reply.
I20260709 13:05:02.297724  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:02.239234  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2067) took 1537 ms. Trace:
I20260709 13:05:02.239550  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:02.241566  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:02.299916  9196 rpcz_store.cc:276] 0709 13:05:00.701495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.701608 (+   113us) service_pool.cc:224] Handling call
0709 13:05:00.710028 (+  8420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.409972 (+699944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.238893 (+828921us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54001d5e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:02.239153 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1458275584}
I20260709 13:05:02.300729  9266 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:02.242730  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:02.301563  9271 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:02.302374  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:02.244226  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:02.304131  9268 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:05:02.245038  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:02.220415 12953 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:02.246404 12869 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40108 (request call id 1039) took 2173 ms. Trace:
I20260709 13:05:02.247366 12915 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:02.248018 12847 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.244266  9270 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:02.320676  9270 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:02.250087  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:02.250190  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:02.253522  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:02.322005  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 71921.5us
W20260709 13:05:02.322184  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 68760.1us
I20260709 13:05:02.203398 12593 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:02.255709  9855 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:05:02.256820 12905 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:05:02.262881  9190 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2060) took 1563 ms (client timeout 1537 ms). Trace:
W20260709 13:05:02.327085  9190 rpcz_store.cc:269] 0709 13:05:00.699247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.700150 (+   903us) service_pool.cc:224] Handling call
0709 13:05:02.262644 (+1562494us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54000eef54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:02.262802 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1882025216}
I20260709 13:05:02.192735 12878 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:05:02.287694  9861 rpcz_store.cc:276] 0709 13:05:00.816254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.819920 (+  3666us) service_pool.cc:224] Handling call
0709 13:05:02.230643 (+1410723us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b5400652554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:02.230854 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345717504}
I20260709 13:05:02.287081  9120 rpcz_store.cc:276] 0709 13:05:01.226622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.226925 (+   303us) service_pool.cc:224] Handling call
0709 13:05:01.227194 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:01.420308 (+193114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.228741 (+808433us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400286254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:02.229020 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442766208}
I20260709 13:05:02.245438 12793 consensus_queue.cc:1059] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:02.303453  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:02.310577  9267 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:05:02.287428 12948 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.191s	user 0.002s	sys 0.000s
I20260709 13:05:02.311937  9266 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:02.319685  9264 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:02.329313  9119 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:02.329331  9268 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:02.285624 12906 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:05:02.335966  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1058) took 1329 ms. Trace:
I20260709 13:05:02.317941 12915 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:02.348661  9859 rpcz_store.cc:276] 0709 13:05:01.006126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.006256 (+   130us) service_pool.cc:224] Handling call
0709 13:05:01.006599 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:01.251168 (+244569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.335670 (+1084502us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400733054. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:02.335904 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":543081856}
I20260709 13:05:02.352113  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:02.324797 12957 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:02.333273 12925 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:02.332515 12877 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:05:02.332741 12963 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:05:02.356266 12847 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:02.357175 12915 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.380532 12915 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:02.358332 12924 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:02.383592 12847 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:02.384240 12847 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.360416 12834 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:02.386464 12834 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:05:02.361769 12902 consensus_meta.cc:287] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.550s	user 0.000s	sys 0.010s
I20260709 13:05:02.364622  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:02.368989  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:02.387686  9190 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:02.388195  9269 raft_consensus.cc:2410] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:05:02.329073 12807 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:02.326983 12905 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:02.371070  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:02.390045  9195 raft_consensus.cc:2376] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11. Re-sending same reply.
I20260709 13:05:02.331195 12888 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.393000  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:02.393826  9856 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:02.396884  9268 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:02.397020  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:02.386948 12834 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:02.400636  9264 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:02.400436 12834 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.401213  9196 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:05:02.395460  9266 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:02.401213  9862 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:05:02.401705  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:02.408277  9267 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:02.410904 12908 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:02.414251  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.414899  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:02.318747 12953 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.420398  9197 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:02.417101  9859 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:02.316234 12869 rpcz_store.cc:276] 0709 13:05:00.072856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.075407 (+  2551us) service_pool.cc:224] Handling call
0709 13:05:00.617730 (+542323us) ts_tablet_manager.cc:934] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
0709 13:05:01.775521 (+1157791us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:05:02.231343 (+455822us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:02.231468 (+   125us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:05:02.245601 (+ 14133us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:05:02.245916 (+   315us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:02.246248 (+   332us) ts_tablet_manager.cc:989] Replica registered.
0709 13:05:02.246314 (+    66us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":21938,"tablet-copy.queue_time_us":399015,"thread_start_us":451884,"threads_started":2}
I20260709 13:05:02.417349 12952 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.421722 12953 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:02.422273 12869 tablet_copy_client.cc:806] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:05:02.422092 12952 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.422781 12877 consensus_queue.cc:1243] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:02.423177 12869 tablet_copy_client.cc:670] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:02.423398 12957 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:02.424306 12957 consensus_queue.cc:1243] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:02.424652 12952 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:02.418562 12898 consensus_queue.cc:1059] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:05:02.418931 12887 consensus_meta.cc:287] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.535s	user 0.005s	sys 0.005s
I20260709 13:05:02.419360 12892 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:02.416378 12755 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.000s
I20260709 13:05:02.418288  9856 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:02.400647  9272 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:02.468302  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:02.468787  9272 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:02.429651  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:02.386579 12847 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:02.474607  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:02.475499  9859 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:02.469200  9196 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
W20260709 13:05:02.479656  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.482503  9859 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:02.439178  9190 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:02.447311  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:02.435843  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:02.484452 12910 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:02.484701  9862 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:02.467361  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:02.485456  9857 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:02.486725  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:02.486852 12685 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.487993 12852 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:02.487785 12685 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.488452 12852 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:02.489418 12852 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.493884 12583 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:02.495055  9862 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:02.495635 12638 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:02.496295 12638 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:02.497473  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 174078us
I20260709 13:05:02.487461 12897 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.498884 12897 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.498879  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:02.501188  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:02.502048 12898 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:02.502655 12898 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:02.503132 12898 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:02.497454 12638 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.508487 12638 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:02.512200 12852 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:02.527596  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.527820  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:02.529913  9858 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:02.472704 12965 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:05:02.478062  9272 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.557672  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:02.558506  9123 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:02.558841 12847 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:02.560505  9272 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:02.476905 12877 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:02.477118 12894 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:02.527011 12953 consensus_queue.cc:1175] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "92919b70789041c8a0de0b3525154fd1" responder_term: 6 status { last_received { term: 6 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 6 index: 7 } }
I20260709 13:05:02.379065 12925 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:02.605760  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:02.588786 12908 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:02.564779 12877 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:02.466542 12887 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.610755 12908 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.471912 12892 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:02.534644 12860 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.564823 12847 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.598346 12894 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.448083 12919 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:05:02.624835  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 132835us
I20260709 13:05:02.640501  9089 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:02.613772 12974 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:02.711673 12892 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:02.652870  9861 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:02.713210 12847 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:02.686159  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 183239us
I20260709 13:05:02.715920 12897 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:02.717173 12974 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:02.718075 12974 consensus_queue.cc:1243] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:05:02.719267 12851 consensus_meta.cc:287] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
I20260709 13:05:02.719081 12898 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.735128  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:02.635267  9272 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:05:02.736236  9196 raft_consensus.cc:2410] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 10 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:05:02.737763  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:02.738024  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:02.738300  9189 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:02.642373  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 167325us
I20260709 13:05:02.739360  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:02.740134  9194 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:02.719774 12851 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.720371  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:02.721069  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:02.721895  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:02.722143 12908 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:02.743125  9862 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:02.743251  9853 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
W20260709 13:05:02.710057  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 69665.4us
I20260709 13:05:02.722973 12751 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:02.726257 12963 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.727515 12983 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:02.621779 12971 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:02.727623  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:02.728826 12860 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:02.730003 12887 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:02.731710 12798 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:02.734436  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:02.773343  9198 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:02.741566  9195 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:02.773718  9164 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:02.746196  9189 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:02.746863  9116 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:02.775460  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:02.776225  9853 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:02.778823  9194 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:02.746290  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:02.669606 12972 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.767786  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:02.622967 12906 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260709 13:05:02.667774 12925 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:02.622092 12964 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
W20260709 13:05:02.782814  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 43720.5us
I20260709 13:05:02.768787  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:02.793388  9164 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:02.768566 12892 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:05:02.784623 12976 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:02.870573  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:02.839809  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 53939.2us
I20260709 13:05:02.871019 12982 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:02.871519 12976 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:02.872313 12976 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:02.823463 12887 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.847452  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 970) took 1127 ms. Trace:
I20260709 13:05:02.781872 12908 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.792253 12964 consensus_queue.cc:1243] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:02.841089  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:02.792068  9192 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:02.825582  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:02.882161  9123 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:02.873972  9855 rpcz_store.cc:276] 0709 13:05:01.720263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.720385 (+   122us) service_pool.cc:224] Handling call
0709 13:05:02.255349 (+534964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.326269 (+ 70920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.847203 (+520934us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b5400652550. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:02.847376 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229454208}
I20260709 13:05:02.784823 12972 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.809996  9853 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:02.794881 12971 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.802747  9196 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
W20260709 13:05:02.850523  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 57219.6us
I20260709 13:05:02.768863 12963 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:02.873360 12887 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.822714 12925 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 13:05:02.789103 12977 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.179s
I20260709 13:05:02.824064 12906 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:03.076992 12957 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:02.923902 12964 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:03.434303 12963 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:02.871676 12982 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.435184 12963 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:03.042080  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 2233) took 1208 ms. Trace:
I20260709 13:05:02.845741 12979 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.593s
I20260709 13:05:03.435228 12971 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:03.436228  9116 rpcz_store.cc:276] 0709 13:05:01.829508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.829645 (+   137us) service_pool.cc:224] Handling call
0709 13:05:03.037216 (+1207571us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54001e4e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:03.037353 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2055055616}
I20260709 13:05:03.174425 12976 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.438231 12971 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:03.438858 12971 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:03.087404 12894 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:02.896111 12908 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:02.844257 12984 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.602s
I20260709 13:05:03.158726 12755 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:02.957898  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.446568 12755 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.315858 12798 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:03.447523  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.447865 12798 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:02.921295  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49858.3us
I20260709 13:05:03.449352 12887 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:03.086133  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:03.450683 12798 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:03.189108  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.208806 12685 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:02.889930 12904 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:02.826370 12981 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:03.456189 12860 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.456275 12904 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.456856 12981 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.456916 12860 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:03.457398 12981 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.458665 12904 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:02.795639 12386 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:03.459339 12386 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:02.980119  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 199548us
I20260709 13:05:02.921082 12898 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.459640 12860 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:03.058000  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 216985us
I20260709 13:05:03.461786  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:02.921269 12952 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:03.464022  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:02.795895 12980 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.670s
W20260709 13:05:03.419374  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 355030us
I20260709 13:05:03.434140 12964 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.076730 12919 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.363863 12973 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:03.474498 12919 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.474874 12973 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.475433 12964 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.034237  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:03.514530  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 480370us
I20260709 13:05:03.515532  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:03.516556 12887 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:03.517030 12887 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:03.517132 12869 tablet_copy_client.cc:538] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:03.517443 12887 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:03.465658  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:03.459764 12386 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:03.470268 12894 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:03.471071  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:03.470928 12925 consensus_queue.cc:1059] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:03.476662 12977 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:03.477718 12919 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.439602 12976 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.485121 12963 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:03.488745  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.443395 12908 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:03.433899 12957 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.502082 12798 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:03.459590 12981 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.509740  9272 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:03.468106 12982 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:02.970551  9123 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:03.558835  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1067) took 1814 ms. Trace:
I20260709 13:05:03.559301 12846 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:03.560885  9235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.006s	sys 0.021s
W20260709 13:05:03.608439  9235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.007s	sys 0.021s
W20260709 13:05:03.608832  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 555347us
I20260709 13:05:03.579005  9197 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:05:03.579625  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.573403 12977 consensus_queue.cc:1243] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 94504d29c6744f38bb06943479df9466 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:05:02.960801 12638 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:03.548498 12834 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:03.066486 12983 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.559653 12847 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:03.439448 12971 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:03.579465  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 482095us
I20260709 13:05:03.606418  9854 rpcz_store.cc:276] 0709 13:05:01.744435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.744717 (+   282us) service_pool.cc:224] Handling call
0709 13:05:01.747836 (+  3119us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:01.837051 (+ 89215us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:01.996780 (+159729us) log.cc:844] Serialized 14 byte log entry
0709 13:05:02.072320 (+ 75540us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:02.072973 (+   653us) log.cc:844] Serialized 52 byte log entry
0709 13:05:02.073371 (+   398us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:02.073645 (+   274us) log.cc:844] Serialized 14 byte log entry
0709 13:05:02.073830 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.108735 (+ 34905us) spinlock_profiling.cc:243] Waited 930 us on lock 0x7b1800457b40. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:03.558490 (+1449755us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b540048c954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:03.558547 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.558594 (+    47us) raft_consensus.cc:1685] finished
0709 13:05:03.558606 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:03.558767 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11598,"prepare.queue_time_us":202219,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":7838,"raft.queue_time_us":17590,"raft.run_cpu_time_us":554,"raft.run_wall_time_us":31199,"spinlock_wait_cycles":155270272,"thread_start_us":173832,"threads_started":2,"wal-append.queue_time_us":84824,"wal-append.run_cpu_time_us":2532,"wal-append.run_wall_time_us":1003778}
W20260709 13:05:03.612238 12902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.003s	sys 0.039s
I20260709 13:05:03.603354 12973 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.646737 12976 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:05:03.677919 12609 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.678702 12609 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.680025 12887 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.681128 12979 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:03.681859 12979 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.682778 12887 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:03.659415 12983 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.684648 12982 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:03.685352 12982 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.686311 12983 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.614373 12893 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:05:03.647840 12977 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.688725 12977 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:03.691210 12977 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:03.616055 12908 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.692121 12908 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.694716 12908 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:03.698473  9267 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:03.704456 12609 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.704818 12886 raft_consensus.cc:3055] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:03.705298 12886 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:03.706000 12979 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.706252 12886 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:03.658383 12902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.003s	sys 0.040s
I20260709 13:05:03.618424 12898 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:03.648841 12892 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:03.716631 12982 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.738430 12981 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.740768 12989 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.749174 12906 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:03.808940 12906 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.750326 12886 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:05:03.818506  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 344950us
I20260709 13:05:03.751224 12964 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.739078 12898 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.751533 12991 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:03.828033  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:03.798575 12892 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.801065 12904 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:03.835217  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:03.823909 12995 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:05:03.806138 12894 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:03.839038  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 373440us
W20260709 13:05:03.844839  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.850433  9089 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:03.852027  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:03.858017  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:03.858510  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:03.858722  9854 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:03.859282  9862 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:03.860782  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:03.861397  9116 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:03.860764  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:03.863991  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:03.864573  9116 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:03.866182  9862 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:03.868546  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:03.869609  9123 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
W20260709 13:05:03.869163  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.871872  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:03.874222  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:03.880838  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:03.838760 12991 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.884191 12755 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.884889 12755 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:03.885341 12755 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:03.885900 12991 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.886301 12755 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.887542 12898 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.859153 12892 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:03.889113  9854 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:03.890443 12892 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:03.812292 12989 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.893136  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:03.862993 12847 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:03.894735 12989 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.859905 12979 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:03.895085 12847 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.896159 12979 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.807349 12981 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:03.907915  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:03.910776  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:03.911342  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:03.899461  9116 raft_consensus.cc:2410] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:05:03.900540  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:03.902164  9192 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:05:03.902874  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:03.916553  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:03.918406  9116 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:03.911458  9194 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:03.870561 12977 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:03.925792  9116 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
W20260709 13:05:03.929342 12985 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:05:03.834923 12964 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:03.929710 12996 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:05:03.898312 12979 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.908167 12386 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.869814 12983 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:03.857656 12904 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.028560  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 658) took 1940 ms. Trace:
I20260709 13:05:04.029699 12904 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:04.030493 12904 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:04.030998 12904 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:03.927280 12898 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.007403  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:04.032054 12898 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.011312  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.029035  9193 rpcz_store.cc:276] 0709 13:05:02.087734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.088648 (+   914us) service_pool.cc:224] Handling call
0709 13:05:02.089586 (+   938us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.089942 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.089975 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.090413 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:05:03.091823 (+1001410us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b180029e960. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:05:03.568992 (+477169us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:03.569058 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.569155 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:03.701511 (+132356us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b48000030d8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:05:04.028307 (+326796us) raft_consensus.cc:1685] finished
0709 13:05:04.028331 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.028496 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":883,"raft.run_cpu_time_us":20281,"raft.run_wall_time_us":1609261,"spinlock_wait_cycles":41309440,"thread_start_us":476863,"threads_started":1,"wal-append.queue_time_us":1657363}
I20260709 13:05:04.033011  9198 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:04.033165 12386 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:04.011646  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.033969  9194 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 5.
I20260709 13:05:04.034149 12898 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:03.929931 12908 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:03.857967 12973 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:05:03.981283 12902 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:04.034860 12908 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.035776 12902 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.037333 12908 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:03.980908 12989 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.037755 12902 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:04.037998 12989 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.039992 13019 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:04.040413 12989 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:04.032564  9116 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:03.993227 12980 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.041716 12980 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.042073  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:04.042915  9116 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:04.042865 12981 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:04.043991  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:04.044626  9116 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:05:04.044836  9855 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:04.045526  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:04.046402  9116 raft_consensus.cc:2410] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:05:04.047210  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:04.048933  9854 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:04.049072  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:04.049717  9116 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:04.049988  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.050558  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:04.051146  9116 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:03.998171 13009 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:04.051890 13009 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:04.052807 12956 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:04.053213 12956 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:04.053269 13009 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:04.054427  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.040526 13019 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:03.891428 12894 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.055038 13019 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:04.051261  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1078) took 1961 ms. Trace:
W20260709 13:05:04.023555 13006 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:05:04.054158 12956 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:04.060437  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.068516  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:04.064746  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.069473  9853 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
W20260709 13:05:04.069834  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 159096us
I20260709 13:05:04.072844  9198 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:04.058007  9860 rpcz_store.cc:276] 0709 13:05:02.089790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.106949 (+ 17159us) service_pool.cc:224] Handling call
0709 13:05:02.107299 (+   350us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:02.150378 (+ 43079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.150408 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:02.151115 (+   707us) log.cc:844] Serialized 78 byte log entry
0709 13:05:02.723720 (+572605us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b1800118ba0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:05:02.951390 (+227670us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:03.214329 (+262939us) log.cc:844] Serialized 14 byte log entry
0709 13:05:03.445559 (+231230us) log.cc:844] Serialized 14 byte log entry
0709 13:05:03.446069 (+   510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:03.446163 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:04.051048 (+604885us) raft_consensus.cc:1685] finished
0709 13:05:04.051069 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.051208 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9198,"prepare.queue_time_us":963360,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":25291,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":96456,"spinlock_wait_cycles":4584192,"thread_start_us":377505,"threads_started":2,"wal-append.queue_time_us":1550888}
I20260709 13:05:04.075372  9194 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:04.076133  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.076854  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.077608  9194 raft_consensus.cc:2368] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8.
I20260709 13:05:04.077739  9853 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:05:04.078794  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.084314  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.086727  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.087446  9192 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:04.089120  9192 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:04.085218  9198 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:05:04.006379 12957 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:04.103992  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.106864  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.107811  9858 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:04.110983  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.111891  9198 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:04.113732  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:04.120340  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.121596  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:04.121775  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.009961 12977 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:04.121235  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.122319  9114 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:04.123364  9198 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:04.121269  9859 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:04.122491  9197 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:04.122571  9193 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:03.998476 12892 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:05:04.128398  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.128592  9116 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:04.133932  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:04.141151  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:04.138471  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.142228  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:04.143755  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.144546  9855 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:04.122905 12898 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.129114 12956 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:04.169708  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:04.123003 12983 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.145831  9114 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:04.161526  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.171307  9860 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:04.146270  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.161338  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:04.122465 13001 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.165977  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.172840  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.173651  9858 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:05:04.175611  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:04.175686  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:04.135032 12846 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:04.176684  9114 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:04.181015  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:04.169442  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.182574  9856 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:04.184036  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:04.182339  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:04.142802 12989 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:04.180154  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.128961 12973 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:05:04.199941  9271 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:04.216128  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:04.240828  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 60229.1us
I20260709 13:05:04.170504  9855 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:04.240844  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 58637.4us
I20260709 13:05:04.241842  9164 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:04.194419  9825 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:04.182750 12979 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:04.256214  9164 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:04.193372 13011 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:04.263666  9238 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:04.260658  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 82860.9us
I20260709 13:05:04.265020  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:04.272432  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:04.276023  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:04.282164  9855 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:04.276362  9266 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:04.195312 12846 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:04.174965 12898 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:04.296579  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.297205  9192 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:04.297573  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.145582 13007 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 13:05:04.298228  9825 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:04.299602  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:04.295904  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:04.301499  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:04.308840  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.310320  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.311141  9858 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:04.184300 12983 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.314828  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:04.323028  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.347777  9164 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:04.349104  9858 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:05:04.298182 12860 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:04.199065 12989 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:04.361339  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:04.350800  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.343633  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.363101  9855 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:05:04.261006 13004 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:04.306073 12887 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:04.371707 12887 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:04.372239  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.372291 12887 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:04.373677  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 59033.7us
I20260709 13:05:04.374591  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:04.304836 13011 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:04.378101  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.301111 12979 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:04.380751  9828 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:04.371155 13004 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.313959 12886 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:04.388057  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:04.391227  9858 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:04.389595  9114 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:04.392911  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.393682  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:04.395627  9087 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:04.316648 13032 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:04.353334 12981 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:04.344519 13016 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:04.314189 12892 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:04.393705  9197 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:04.359848 12860 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.273762 12977 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:04.420881 12860 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:04.420343  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.388795 12834 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:04.410671  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:04.384338 12888 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:04.183192 13001 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.386636 13024 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:05:04.396772 12964 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:04.396559 13004 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:04.421267 13029 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:04.436771 13018 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:05:04.392234 13017 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.209s
W20260709 13:05:04.554692  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.907634  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 478992us
W20260709 13:05:04.525723  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 178394us
I20260709 13:05:04.900563  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:04.910859  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:04.911720  9271 raft_consensus.cc:2410] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:04.911769  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:04.912505  9267 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:05:04.635499 13019 consensus_meta.cc:287] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.580s	user 0.006s	sys 0.005s
I20260709 13:05:04.576668 13011 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.914321 13019 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:04.578217  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.766350  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:04.480419  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 69858.6us
I20260709 13:05:04.505127  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:04.907657  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:04.434389 12989 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
W20260709 13:05:05.228703 13032 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.005s	sys 0.019s
W20260709 13:05:05.229657  9238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.004s	sys 0.017s
W20260709 13:05:05.230077  9238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.004s	sys 0.017s
W20260709 13:05:04.437806 12999 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.395s	user 0.000s	sys 0.001s
W20260709 13:05:04.631309 13038 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:05:04.566629 12869 tablet_bootstrap.cc:492] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
W20260709 13:05:04.898355 13051 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:05:04.906637 12887 consensus_meta.cc:287] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.004s
I20260709 13:05:04.904201 13030 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:05.290179  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:05.252209 12977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.013s	sys 0.046s
W20260709 13:05:04.524168 13020 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 13:05:04.435142 13023 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:05:05.254590  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:05.309520  9858 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
W20260709 13:05:04.521176 13033 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:05:04.416208 13016 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.431345 12983 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:04.455351 12834 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.324699  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1086) took 1259 ms. Trace:
W20260709 13:05:05.296988 12977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.014s	sys 0.046s
I20260709 13:05:05.326151  9861 rpcz_store.cc:276] 0709 13:05:04.065199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.065324 (+   125us) service_pool.cc:224] Handling call
0709 13:05:04.065691 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:04.066086 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.066116 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:04.066623 (+   507us) log.cc:844] Serialized 26 byte log entry
0709 13:05:04.306389 (+239766us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:04.306428 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.306494 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:05.324402 (+1017908us) raft_consensus.cc:1685] finished
0709 13:05:05.324432 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.324619 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":768,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":57326,"thread_start_us":169028,"threads_started":1,"wal-append.queue_time_us":331830}
I20260709 13:05:04.673439 13053 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:04.479666 12755 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:04.429847 12990 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:04.906510 12888 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:05.187659 13029 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.333976 12755 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:04.509550 12908 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:04.474206 13031 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:04.480641 12981 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:04.611418 12877 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:04.694856 13047 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:04.519265 13026 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:04.434592 12980 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:04.522776 12904 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:05.243541 13032 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.005s	sys 0.019s
I20260709 13:05:04.438022 12892 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.380302 13047 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:04.418519 12846 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:05.297799 12860 consensus_meta.cc:287] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.871s	user 0.010s	sys 0.000s
I20260709 13:05:04.432097 12979 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:05.330701  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.329030 12834 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.342662 13060 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.342777 12888 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:05.343650 12755 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:05.334167 13053 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.290963 12887 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.323244 13016 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:05.343395 13029 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:05.334591 13017 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.355227 12908 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:05.356457 12898 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:05.354445 12990 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:05.358619 13031 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:05.365090 13019 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:05.376092 12956 consensus_meta.cc:287] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.200s	user 0.007s	sys 0.006s
I20260709 13:05:05.307281 13030 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:05.275542 13013 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:05:05.371080 12981 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.436033 13017 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:05.425882 13057 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
I20260709 13:05:05.448156 13030 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:05.448891 13064 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:05.450142 13046 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:05:05.451053 13061 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:05:05.456964 13047 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:05.433787 12908 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:05.439934 13031 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:05.498492 13017 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:05.500430 12981 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.492643 13064 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.433787 13060 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.496217 13053 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.495990 12990 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.519295 13060 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.521746 13064 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.501276 13065 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.548743 13065 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.559856 13065 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.568563 13004 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.579849 13004 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.580351 13029 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.584148 13029 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:05.630964  9087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.234s	user 0.006s	sys 0.013s
W20260709 13:05:04.643375  9117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.459s	user 0.012s	sys 0.033s
I20260709 13:05:04.681196  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2246) took 2282 ms. Trace:
W20260709 13:05:05.632376  9118 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.227s	user 0.005s	sys 0.021s
W20260709 13:05:05.636009  9164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.003s	sys 0.003s
W20260709 13:05:05.636976  9195 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.132s	user 0.036s	sys 0.059s
W20260709 13:05:05.637562  9198 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.004s	sys 0.006s
I20260709 13:05:05.637725  9114 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:04.569209  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2249) took 2135 ms. Trace:
W20260709 13:05:05.640416  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.015s	sys 0.035s
I20260709 13:05:05.643846  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2275) took 1753 ms. Trace:
I20260709 13:05:05.647539  9266 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:05.572434 13001 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:05.748168  9195 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.243s	user 0.037s	sys 0.059s
I20260709 13:05:05.594592 13031 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.748505 12893 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:05:05.744990  9122 rpcz_store.cc:276] 0709 13:05:03.890293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.567934 (+677641us) service_pool.cc:224] Handling call
0709 13:05:04.568209 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.639701 (+1071492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.639797 (+    96us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.639878 (+    81us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:05.639932 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.643220 (+  3288us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001905d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:05.643271 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.643555 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2462932864}
W20260709 13:05:05.748049  9164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.004s	sys 0.003s
I20260709 13:05:05.750885  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:05.681272  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2466) took 1260 ms. Trace:
I20260709 13:05:05.751916  9264 rpcz_store.cc:276] 0709 13:05:04.420857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.254748 (+833891us) service_pool.cc:224] Handling call
0709 13:05:05.255025 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.647706 (+392681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.647770 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.647813 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:05.647848 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.680725 (+ 32877us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54000ea6d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:05.680780 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.681029 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901614464}
I20260709 13:05:05.752631  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.753733  9122 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:05.754532  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:05.754889  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:05.681910 12851 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.684417 12972 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.748095  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 648) took 1750 ms. Trace:
I20260709 13:05:05.696152 13054 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:05.701757  9828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.007s	sys 0.013s
W20260709 13:05:05.702335  9825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.008s	sys 0.004s
I20260709 13:05:05.703500 12956 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.704030 13016 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.411811 12860 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.705226 12892 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.706235 13052 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:05.778458 13052 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:05.781337 13052 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:05.782760 13004 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:05.784015  9117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.587s	user 0.014s	sys 0.041s
I20260709 13:05:05.785892  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2270) took 2722 ms. Trace:
I20260709 13:05:05.710311  9270 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.725514  9089 leader_election.cc:400] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8.
I20260709 13:05:05.729329 12977 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:05.729898  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:05.730793  9272 raft_consensus.cc:2410] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:05.768260  9122 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:05.725916 12991 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:05.770797 12983 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:05.771157  9828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.007s	sys 0.014s
I20260709 13:05:05.771311 13031 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:05.771476  9825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.009s	sys 0.005s
W20260709 13:05:05.772966  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.016s	sys 0.035s
W20260709 13:05:05.773296  9087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.377s	user 0.007s	sys 0.013s
W20260709 13:05:05.747972  9118 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.349s	user 0.006s	sys 0.021s
I20260709 13:05:05.768908  9114 rpcz_store.cc:276] 0709 13:05:03.997828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.613018 (+615190us) service_pool.cc:224] Handling call
0709 13:05:05.636710 (+1023692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.744567 (+107857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.748016 (+  3449us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:05.783876  9163 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.008s	sys 0.008s
I20260709 13:05:05.722572 12887 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:05.787313  9089 leader_election.cc:403] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:05:05.747946  9269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 805) took 3480 ms (client timeout 2184 ms). Trace:
I20260709 13:05:05.786365  9117 rpcz_store.cc:276] 0709 13:05:03.063059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.176639 (+113580us) service_pool.cc:224] Handling call
0709 13:05:03.177043 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.177287 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:03.184124 (+  6837us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.785135 (+2601011us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.785288 (+   153us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:05.785435 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.785582 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.785810 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":228752,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"thread_start_us":2600293,"threads_started":1,"wal-append.queue_time_us":695094}
I20260709 13:05:05.790292  9116 raft_consensus.cc:2376] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13. Re-sending same reply.
I20260709 13:05:05.794057 12905 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:05.797011 12988 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.991s	user 0.018s	sys 0.015s
I20260709 13:05:05.797363 12956 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.769508  9120 rpcz_store.cc:276] 0709 13:05:02.398919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.403150 (+  4231us) service_pool.cc:224] Handling call
0709 13:05:02.403449 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:02.579034 (+175585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.579077 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:04.068397 (+1489320us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b2c00000f48. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:05:04.068974 (+   577us) log.cc:844] Serialized 52 byte log entry
0709 13:05:04.300984 (+232010us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:04.301023 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.301089 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:04.680802 (+379713us) raft_consensus.cc:1685] finished
0709 13:05:04.680822 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.681098 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":76856,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":948,"spinlock_wait_cycles":25604864,"thread_start_us":310603,"threads_started":2,"wal-append.queue_time_us":545007,"wal-append.run_cpu_time_us":2276,"wal-append.run_wall_time_us":1004368}
I20260709 13:05:05.800066 13016 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:05.800588  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.772486 12893 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:05.800829  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1096) took 1931 ms. Trace:
I20260709 13:05:05.801702 12979 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:05.775996  9198 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.004s	sys 0.006s
I20260709 13:05:05.807123  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:05.809566  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 84598.4us
W20260709 13:05:05.807423  9269 rpcz_store.cc:269] 0709 13:05:02.239214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.239328 (+   114us) service_pool.cc:224] Handling call
0709 13:05:05.685471 (+3446143us) spinlock_profiling.cc:243] Waited 87.7 ms on lock 0x7b54003a25d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:05.719765 (+ 34294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201738624}
I20260709 13:05:05.829409  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.831670  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:05.833101  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:05.833797  9269 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:05.804056 12991 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:05.834229  9121 rpcz_store.cc:276] 0709 13:05:02.434238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.434367 (+   129us) service_pool.cc:224] Handling call
0709 13:05:02.450735 (+ 16368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:02.672799 (+222064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.672852 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:02.672913 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:02.672957 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:04.568977 (+1896020us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b540034e754. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:04.569019 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:04.569145 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510136704}
I20260709 13:05:05.824302  9862 rpcz_store.cc:276] 0709 13:05:03.869538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.930984 (+ 61446us) service_pool.cc:224] Handling call
0709 13:05:03.931909 (+   925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.033940 (+102031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:04.033978 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:04.034029 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:04.034059 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.800247 (+1766188us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54007bc5d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:05.800318 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.800597 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234020608}
I20260709 13:05:05.835265 12991 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:05.845665 12998 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 2.006s	user 0.010s	sys 0.026s
I20260709 13:05:05.846443 12977 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:05.848341 12979 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.851405 12979 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:05.886322  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.880172 12982 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:05.880100  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:05.891292  9862 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:05.892541  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:05.893827 12990 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.894526 12990 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.896685 12888 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.858625  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.902531 12798 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:05.903507 12798 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:05.902477  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.905941  9120 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:05.907264 12977 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:05.847347 12834 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.826180 13004 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.896288 12908 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.843829 12892 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8.
I20260709 13:05:05.857609  9164 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:05.852702  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.865305  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:05.851987  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 1.45634e+06us
I20260709 13:05:05.824142 12893 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:05.917999  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:05.860723 13073 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.856825  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:05.872090 12957 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.871493 13016 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:05.876113 12905 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:05.861478 12956 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:05.859373  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.868705  9264 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
W20260709 13:05:05.828503  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 904197us
I20260709 13:05:05.851982 12983 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:05.859063  9266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1916) took 1596 ms (client timeout 1737 ms). Trace:
W20260709 13:05:05.853915 13082 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:05:05.772535 13054 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:05.891077 12982 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:05.897249 12990 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.856087 12851 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:05.904968  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 809) took 3661 ms. Trace:
I20260709 13:05:05.866355  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 673) took 1570 ms. Trace:
I20260709 13:05:05.876906  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2463) took 1543 ms. Trace:
I20260709 13:05:05.907711  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:05.925136  9122 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
W20260709 13:05:05.955139  9193 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 672) took 1653 ms (client timeout 2165 ms). Trace:
I20260709 13:05:05.919703  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2072) took 3185 ms. Trace:
I20260709 13:05:05.919298  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:05.934252 13004 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:05.936650 12866 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:05:05.937830 13016 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.939378  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:05.920953 12755 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:05.939992  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2455) took 1735 ms (client timeout 1848 ms). Trace:
I20260709 13:05:05.920437 12908 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:05.941250 13041 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.909279 12924 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:05.948105 12971 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:05.946113  9266 rpcz_store.cc:269] 0709 13:05:04.262143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.301282 (+ 39139us) service_pool.cc:224] Handling call
0709 13:05:05.858739 (+1557457us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54000eb0d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:05.858963 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3086410624}
W20260709 13:05:05.965777  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 108419us
I20260709 13:05:05.909912  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2244) took 3773 ms. Trace:
I20260709 13:05:05.957306  9191 rpcz_store.cc:276] 0709 13:05:02.734364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.734525 (+   161us) service_pool.cc:224] Handling call
0709 13:05:03.136290 (+401765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:03.657101 (+520811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.657133 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:03.657175 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:03.657204 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.917956 (+2260752us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b540010cf54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:05.918024 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.919298 (+  1274us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b540010cf50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:05.919481 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2118418432}
I20260709 13:05:05.955379 12977 raft_consensus.cc:2764] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:05:05.949184  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 92615.2us
W20260709 13:05:05.957157  9193 rpcz_store.cc:269] 0709 13:05:04.295497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.295636 (+   139us) service_pool.cc:224] Handling call
0709 13:05:05.948211 (+1652575us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54000eef54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:05.948390 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3446517120}
W20260709 13:05:05.963457  9272 rpcz_store.cc:269] 0709 13:05:04.204435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.204588 (+   153us) service_pool.cc:224] Handling call
0709 13:05:05.939754 (+1735166us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54000fb4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:05.939929 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3205504896}
I20260709 13:05:05.959852 12834 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.974627  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.969133  9118 rpcz_store.cc:276] 0709 13:05:02.136577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.153434 (+ 16857us) service_pool.cc:224] Handling call
0709 13:05:02.178682 (+ 25248us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:02.179079 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:03.184866 (+1005787us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b18003eb000. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:04.057245 (+872379us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b18003eb000. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f4455 00007f765a0dc3c7 00007f76566ee9a7 00007f76566fe0bf 00007f76566fe038
0709 13:05:04.260483 (+203238us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.830145 (+1569662us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:05.853434 (+ 23289us) log.cc:844] Serialized 52 byte log entry
0709 13:05:05.902209 (+ 48775us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:05.902278 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.909629 (+  7351us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:05.909684 (+    55us) raft_consensus.cc:1685] finished
0709 13:05:05.909697 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.909827 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3163,"prepare.queue_time_us":19102,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":82017,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":6554,"spinlock_wait_cycles":8075392,"thread_start_us":1700127,"threads_started":3,"wal-append.queue_time_us":987000}
I20260709 13:05:05.980098 12851 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.981581  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.982420  9118 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:05.982805  9862 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:05.983064 12851 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:05.983356  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.985576  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.983820  9120 raft_consensus.cc:2376] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11. Re-sending same reply.
I20260709 13:05:05.953224  9267 rpcz_store.cc:276] 0709 13:05:04.301584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.913090 (+611506us) service_pool.cc:224] Handling call
0709 13:05:04.913463 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.648724 (+735261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:05.648790 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.650006 (+  1216us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.674552 (+ 24546us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:05.786714 (+112162us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.812643 (+ 25929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.826050 (+ 13407us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:05.837918 (+ 11868us) raft_consensus.cc:1685] finished
0709 13:05:05.837940 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.844589 (+  6649us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":763508,"raft.queue_time_us":414273,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":48419,"thread_start_us":506009,"threads_started":3,"wal-append.queue_time_us":42187}
I20260709 13:05:05.991211  9269 raft_consensus.cc:2410] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:05.952858  9198 rpcz_store.cc:276] 0709 13:05:04.295821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.295979 (+   158us) service_pool.cc:224] Handling call
0709 13:05:04.296371 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.296572 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:04.372515 (+ 75943us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.865736 (+1493221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.865849 (+   113us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:05.865942 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.866057 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.866223 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":400356,"prepare.queue_time_us":312723,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":505140,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":1122428,"threads_started":3,"wal-append.queue_time_us":411497}
I20260709 13:05:05.992304 12849 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:05.992218  9828 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:05.992856 12755 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:05.993695  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.993161 13041 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.994138 12908 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:05.952745  9265 rpcz_store.cc:276] 0709 13:05:02.243341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.243590 (+   249us) service_pool.cc:224] Handling call
0709 13:05:02.243934 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:02.244347 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:02.244378 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:02.244862 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:05:02.335850 (+ 90988us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:02.373340 (+ 37490us) log.cc:844] Serialized 14 byte log entry
0709 13:05:02.561911 (+188571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:02.562436 (+   525us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:05.892068 (+3329632us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b18003e88a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:05.904374 (+ 12306us) raft_consensus.cc:1685] finished
0709 13:05:05.904552 (+   178us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:05.904742 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5272,"prepare.queue_time_us":40730,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":86668,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":2133,"spinlock_wait_cycles":65508992,"thread_start_us":278845,"threads_started":2,"wal-append.queue_time_us":135005}
W20260709 13:05:05.994915  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 94948.1us
I20260709 13:05:06.009543  9115 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6. Ops: []
W20260709 13:05:05.988471  9825 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.552s (SENT)
I20260709 13:05:06.002974  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 677) took 1706 ms. Trace:
I20260709 13:05:06.013356  9114 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:06.003921  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:05.986588 12833 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:06.026482  9120 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:05:06.026490  9122 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:06.035703 12833 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.036617  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2465) took 1650 ms. Trace:
I20260709 13:05:06.038352 12833 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:06.039819  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:05.971463 13073 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:06.037124  9271 rpcz_store.cc:276] 0709 13:05:04.386215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.913246 (+527031us) service_pool.cc:224] Handling call
0709 13:05:04.913538 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:05.651490 (+737952us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:05:05.651557 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:05.652357 (+   800us) log.cc:844] Serialized 26 byte log entry
0709 13:05:05.678960 (+ 26603us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:05.679000 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.688415 (+  9415us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.033526 (+345111us) raft_consensus.cc:1685] finished
0709 13:05:06.033552 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.036422 (+  2870us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":319163,"raft.queue_time_us":788790,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":324,"thread_start_us":444068,"threads_started":2,"wal-append.queue_time_us":82556}
I20260709 13:05:06.012934 12893 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.049742 13016 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:06.050228 12893 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:06.051378 12982 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:05.994498 13004 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:05.994663 12908 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.992910 12849 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.015013 13054 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:06.021114  9195 rpcz_store.cc:276] 0709 13:05:04.297038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.298041 (+  1003us) service_pool.cc:224] Handling call
0709 13:05:04.300952 (+  2911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:04.301165 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:04.504596 (+203431us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.943105 (+1438509us) log.cc:844] Serialized 14 byte log entry
0709 13:05:05.964060 (+ 20955us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:05.967181 (+  3121us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:05.974050 (+  6869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:05.986882 (+ 12832us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.002907 (+ 16025us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2595,"prepare.queue_time_us":563063,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":3650,"raft.run_cpu_time_us":353,"raft.run_wall_time_us":3451,"thread_start_us":1624682,"threads_started":3,"wal-append.queue_time_us":974557}
I20260709 13:05:06.010454 12866 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:05.987546 12979 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:06.010044 12971 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:06.010455 13021 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:05:06.070739  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:06.057801 12849 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:06.056553 12908 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:06.060472 12992 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:06.074854 12992 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:05.983950 12957 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.078109 12957 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.079833  9272 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:06.080776 12983 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:06.081079  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:06.081911  9272 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:06.082360  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:06.083021  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:06.083762  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:06.085110  9089 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:06.086063 13054 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:06.086615 13054 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.086737  9089 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:06.086380 12866 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.087157 13054 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:06.071591  9118 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
W20260709 13:05:06.073951  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 57924.5us
I20260709 13:05:06.090022 12866 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.090714 12924 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:06.091424  9164 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:06.092079 12971 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:06.093958 13070 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:06.094378 13070 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.094643 12971 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:06.095487 13004 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:06.094732 13070 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:06.098830 13004 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:06.074211  9114 raft_consensus.cc:2410] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:06.100313  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:06.103976 12979 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.105551 12977 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:06.106067 12977 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.106509 12977 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:06.107941  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 45545.5us
I20260709 13:05:06.091389 12924 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:06.115341 12983 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:06.127233 12976 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:06.116901 12888 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:06.138823  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 163028us
I20260709 13:05:06.125527 12983 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.144511 12976 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.145428 12976 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.147759  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:06.148684  9272 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:06.147773 12983 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.155934 12877 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.162022 13012 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:06.162825 13012 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.163937 13021 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:06.164623 13021 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.165382 13012 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:06.165983 12849 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.166965 13021 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:06.168078  9119 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 13:05:06.168159  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 81568.6us
I20260709 13:05:06.168587  9119 pending_rounds.cc:85] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 4
I20260709 13:05:06.168876  9119 pending_rounds.cc:107] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:05:06.169175  9119 raft_consensus.cc:2889] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:06.170106 12991 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:06.171125  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:06.172111 13008 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.172703 13008 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:06.174060  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2245) took 3775 ms. Trace:
I20260709 13:05:06.174297  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.174652  9119 rpcz_store.cc:276] 0709 13:05:02.398574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.398698 (+   124us) service_pool.cc:224] Handling call
0709 13:05:02.398978 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.169794 (+3770816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.171495 (+  1701us) spinlock_profiling.cc:243] Waited 3.77 s on lock 0x7b5400267354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:06.173956 (+  2461us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8666685056}
I20260709 13:05:06.176358 12755 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:06.176292  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.177745  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.178007  9857 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:06.178509  9858 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:06.178685  9855 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:06.179881  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.179488 12908 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.180524  9858 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:05:06.166396 12976 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.184629 12888 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:06.185343 12888 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:06.188884 12925 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.490s	user 0.012s	sys 0.048s
I20260709 13:05:06.189473 13008 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:06.147822  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1901) took 4539 ms. Trace:
W20260709 13:05:06.194020  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49475.8us
W20260709 13:05:06.200001  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 52657.2us
W20260709 13:05:06.189474 12925 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.616s	user 0.013s	sys 0.048s
W20260709 13:05:06.163945  9826 connection.cc:256] RPC call timeout handler was delayed by -3.3761178650000829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
W20260709 13:05:06.209064  9826 connection.cc:256] RPC call timeout handler was delayed by -3.1727178650000951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
W20260709 13:05:06.209407  9826 connection.cc:256] RPC call timeout handler was delayed by -2.8889349800000446s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
W20260709 13:05:06.209681  9826 connection.cc:256] RPC call timeout handler was delayed by -2.7539349800000537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 13:05:06.210124  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.211071  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.212363  9268 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:05:06.212749  9268 pending_rounds.cc:85] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 8
I20260709 13:05:06.212980  9268 pending_rounds.cc:107] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:05:06.213236  9268 raft_consensus.cc:2889] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:06.214025 13022 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.214761  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1911) took 3099 ms. Trace:
I20260709 13:05:06.214630 13022 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.215564  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:06.215960  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:06.216755 12966 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:06.217518 12966 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:06.217911 12966 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.218359 12966 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:06.218453  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:06.210098  9826 connection.cc:256] RPC call timeout handler was delayed by -2.4963550610000311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
I20260709 13:05:06.220587  9272 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
W20260709 13:05:06.225972  9826 connection.cc:256] RPC call timeout handler was delayed by -2.4074349799999482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
I20260709 13:05:06.215155  9268 rpcz_store.cc:276] 0709 13:05:03.115556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.115728 (+   172us) service_pool.cc:224] Handling call
0709 13:05:05.856213 (+2740485us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:05:05.856508 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.213751 (+357243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.214554 (+   803us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54000fb9d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:06.214705 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":829941888}
I20260709 13:05:06.190979 12888 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:06.217782 13022 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.231433  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:06.232688  9855 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:06.232785 12800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:06.233530  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:06.235224  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.249320  9858 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
W20260709 13:05:06.249660  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 62122.1us
I20260709 13:05:06.140642 13054 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.263783 13054 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.209566 12908 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:06.255573  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:06.256402  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.256970  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.294517  9857 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:05:06.257925  9856 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:06.258728  9087 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.236s (SENT) [suppressed 15 similar messages]
I20260709 13:05:06.261289  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:06.261889 13021 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.264237 12924 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:06.302775  9853 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:06.252699 12860 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.270594  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:06.293228  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:06.271034 12952 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.271912  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.276518 12886 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:06.280328  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:06.281795 13008 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:06.311199  9193 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:06.311420 13008 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.283141  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:06.313551  9121 raft_consensus.cc:2410] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:06.288064  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:06.289718  9116 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:05:06.290277  9270 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.314505  9116 pending_rounds.cc:85] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 6
I20260709 13:05:06.314810  9116 pending_rounds.cc:107] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:05:06.315049  9116 raft_consensus.cc:2889] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:06.304087  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.308423  9191 raft_consensus.cc:2368] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14.
I20260709 13:05:06.316704  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.279341 12977 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:06.190102  9263 rpcz_store.cc:276] 0709 13:05:01.516660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:01.516818 (+   158us) service_pool.cc:224] Handling call
0709 13:05:01.538322 (+ 21504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:01.555551 (+ 17229us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:01.564308 (+  8757us) log.cc:844] Serialized 14 byte log entry
0709 13:05:01.818563 (+254255us) spinlock_profiling.cc:243] Waited 5.1 ms on lock 0x7b18003a76e0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535c52a6 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1
0709 13:05:01.819152 (+   589us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b18003a76e0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:03.623267 (+1804115us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b18003a76e0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:05:04.012497 (+389230us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:05.692697 (+1680200us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b2c00112948. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:05:05.693737 (+  1040us) log.cc:844] Serialized 52 byte log entry
0709 13:05:05.739038 (+ 45301us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:05.988979 (+249941us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.024627 (+ 35648us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.024938 (+   311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.055537 (+ 30599us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.055658 (+   121us) raft_consensus.cc:1685] finished
0709 13:05:06.055670 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.056036 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32032,"prepare.queue_time_us":396186,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":2431,"raft.queue_time_us":183712,"raft.run_cpu_time_us":568,"raft.run_wall_time_us":45178,"spinlock_wait_cycles":24554752,"thread_start_us":593349,"threads_started":3,"wal-append.queue_time_us":2585067,"wal-append.run_cpu_time_us":1103,"wal-append.run_wall_time_us":1135030}
I20260709 13:05:06.322660  9861 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:05:06.324242  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 58550.5us
I20260709 13:05:06.324151 13013 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:06.328234  9855 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:06.330565  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:06.331861  9858 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:06.332000  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:06.333324  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:06.333552  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:06.334024  9189 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.341988 12851 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:06.342510 12851 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:06.342890 12851 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:06.320727 12886 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:06.344755  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:06.345715 12924 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:06.346257 12924 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.346707 12924 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:06.348168  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 132748us
I20260709 13:05:06.319357 12952 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:06.354480  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:06.325030  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:06.357251  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.374251  9857 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:06.377270  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.379199  9856 raft_consensus.cc:2410] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:06.378994  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:06.382439  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 61600.5us
I20260709 13:05:06.379688  9164 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:06.385281  9860 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:05:06.385432  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 50585us
I20260709 13:05:06.386416  9161 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:06.384964  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 123761us
I20260709 13:05:06.355438 12908 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:06.403415  9270 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:05:06.400322  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.356174 12849 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.404635  9828 leader_election.cc:400] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14.
I20260709 13:05:06.408723  9828 leader_election.cc:403] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:05:06.407924  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.411262  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 810) took 4167 ms. Trace:
I20260709 13:05:06.412948  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:06.414427  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:06.415614 12887 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:06.416070 12887 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:06.416482 12887 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:06.418779 13000 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.419438 13000 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.423928 13045 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:06.424525 13045 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:06.411641  9270 rpcz_store.cc:276] 0709 13:05:02.243638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:02.243770 (+   132us) service_pool.cc:224] Handling call
0709 13:05:02.244036 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:05.709205 (+3465169us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54003f7620. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f765666206f 00007f76566daf60 00007f76566dc7de 00007f76566e6b49 00007f76566e7416 00007f76566e3d8f 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62
0709 13:05:06.290060 (+580855us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b58008f27d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566400aa 00007f76566c2447 00007f76566dafe7 00007f76566dc7de 00007f76566e6b49 00007f76566e7416 00007f76566e3d8f 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4
0709 13:05:06.403826 (+113766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.411206 (+  7380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":667164288}
W20260709 13:05:06.433532  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:06.436311  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 84180.6us
I20260709 13:05:06.437762  9189 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:06.506209 12851 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:06.600165  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.384922 13070 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.513587  9855 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
W20260709 13:05:06.508906  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 79722us
I20260709 13:05:06.424973 13045 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:06.585901  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.592474  9193 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:06.635499  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:06.653244  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:06.672699  9861 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:05:06.679319  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 88759.8us
W20260709 13:05:06.656961 12983 consensus_meta.cc:287] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.000s
W20260709 13:05:06.719604  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 189373us
I20260709 13:05:06.724975  9858 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:06.730821  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.727422  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:06.739395 12886 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.732937 12983 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:06.741690  9197 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:06.732501 12834 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:06.743336  9862 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:06.743623 12834 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:06.744138 12834 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:06.747803 12850 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:06.748178  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2054) took 3170 ms. Trace:
I20260709 13:05:06.748572  9196 rpcz_store.cc:276] 0709 13:05:03.578067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:03.578232 (+   165us) service_pool.cc:224] Handling call
0709 13:05:03.578678 (+   446us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:03.579154 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:03.579194 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:03.580426 (+  1232us) log.cc:844] Serialized 52 byte log entry
0709 13:05:03.905695 (+325269us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:04.302101 (+396406us) log.cc:844] Serialized 14 byte log entry
0709 13:05:06.690915 (+2388814us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b180023f080. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f765628c4bc 00007f76535da48b 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287
0709 13:05:06.747469 (+ 56554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.747586 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:06.747649 (+    63us) raft_consensus.cc:1685] finished
0709 13:05:06.747786 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:06.747995 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16949,"prepare.queue_time_us":703291,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":26392,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":43494,"spinlock_wait_cycles":7508736,"thread_start_us":614786,"threads_started":3,"wal-append.queue_time_us":403614,"wal-append.run_cpu_time_us":2984,"wal-append.run_wall_time_us":1393741}
I20260709 13:05:06.748584 12850 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:06.752382  9862 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:05:06.769937  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 46782us
I20260709 13:05:06.646068 12583 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.135s
W20260709 13:05:06.786625  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59324.1us
I20260709 13:05:06.785897  9855 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:06.810947  9162 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:06.812254  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 425959us
I20260709 13:05:06.804797  9857 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:06.815930 12851 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.832329  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:06.800423 13013 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:06.837885  9164 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:06.837190  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 423216us
I20260709 13:05:06.816669 13012 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14.
W20260709 13:05:06.836458 12966 consensus_meta.cc:287] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.000s
I20260709 13:05:06.838562  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:06.852708 12609 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:06.854585  9853 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:06.861487 12807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.035s	sys 0.094s
I20260709 13:05:06.820966 13008 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.849989 12860 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:06.872598 13065 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:06.873162 13065 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:06.874168 13065 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.875927 12910 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:05:06.880770  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:06.883384 12990 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:06.886533  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 82384.7us
I20260709 13:05:06.872120 13070 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.867908 13010 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:06.889591  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.890044  9857 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:06.854007 12925 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:06.834666 12849 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 13:05:06.863533 12807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.036s	sys 0.095s
I20260709 13:05:06.859059 13013 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:06.897286  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:06.898976  9858 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:06.895720  9853 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:06.865370 12886 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.864955 12609 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:06.902619  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:06.902833  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.904119  9857 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:06.904500  9858 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:05:06.897116 12952 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.543s	user 0.004s	sys 0.008s
I20260709 13:05:06.906066  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:06.906245 13013 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:06.907883 13013 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:06.910096  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:06.910777  9855 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:05:06.889329 12963 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:06.859771 12966 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.925531  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:06.901683  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:06.942624 12609 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.906114 12952 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:06.941192 12892 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.947219 12886 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:06.950702  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 536333us
I20260709 13:05:06.967893 13011 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.956679 13045 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:06.975318  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 73699.6us
W20260709 13:05:06.962782 12887 consensus_meta.cc:287] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.002s
I20260709 13:05:06.937471 12925 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:06.971390 12963 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.989795  9194 raft_consensus.cc:2368] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6.
W20260709 13:05:06.991030 13021 consensus_meta.cc:287] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.673s	user 0.006s	sys 0.005s
I20260709 13:05:06.991518 13021 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:06.991645 13011 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:06.992369 13021 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:06.979929 12991 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:06.984126 12887 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.010298 12887 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:07.010522 12924 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.663s	user 0.009s	sys 0.001s
I20260709 13:05:07.003712 12925 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:07.025704 13111 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:05:07.035105  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 58514.6us
I20260709 13:05:06.985399 12892 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.007001 13021 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:07.043834  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:07.000645 12991 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:07.058770 12991 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:07.059386 12991 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.011197 12924 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.052848  9271 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:05:07.059834 12966 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.002568 12963 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.069998  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:07.075237  9268 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:07.076354  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:07.077170  9263 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:07.091830  9266 raft_consensus.cc:3055] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:07.092267  9266 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:07.087697 12991 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.101850  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:07.102717  9271 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:05:07.114012  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 279767us
I20260709 13:05:07.115963  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:07.113405  9266 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.118394  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:07.119064  9857 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:07.042807 13008 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:05:07.129947  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:07.131050  9858 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 10: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:07.139389  9266 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:07.122394 12800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40124 (request call id 944) took 9720 ms. Trace:
I20260709 13:05:07.151000  9164 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:07.156049  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 236872us
I20260709 13:05:07.134611 13045 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:07.160877  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 277173us
W20260709 13:05:07.163537  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.171092  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.189332  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.185895  9857 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:07.192920  9116 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:07.219668  9164 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:07.177520  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.224366  9268 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:07.227306  9266 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:05:07.228363  9266 pending_rounds.cc:85] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 9
I20260709 13:05:07.228900  9266 pending_rounds.cc:107] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:05:07.229377  9266 raft_consensus.cc:2889] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:07.177794 12953 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:07.188621 13114 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.179847 12892 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.178149 12991 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.192833 13111 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.245476  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:07.246402 13106 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:05:07.246646  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:07.246848 13118 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:07.275141 13118 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:07.275931 13113 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:05:07.277467 13118 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:07.225821 12886 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:07.225652 12925 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.278728 12925 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.280570 13013 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.280856 12925 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.281234 13013 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.153849 12800 rpcz_store.cc:276] 0709 13:04:57.369880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:57.371300 (+  1420us) service_pool.cc:224] Handling call
0709 13:04:58.467227 (+1095927us) ts_tablet_manager.cc:934] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
0709 13:05:06.231092 (+7763865us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:05:06.882986 (+651894us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:06.883106 (+   120us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:05:06.976180 (+ 93074us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:05:07.007946 (+ 31766us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:07.064410 (+ 56464us) ts_tablet_manager.cc:989] Replica registered.
0709 13:05:07.088416 (+ 24006us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":48801,"tablet-copy.queue_time_us":900303,"thread_start_us":504261,"threads_started":2}
I20260709 13:05:07.254472  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.254664  9122 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:07.283905  9114 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:07.284587  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.285404  9122 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:05:07.286265  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.287345  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.254966  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.255228  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.256283 12963 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.256779 12924 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.289112  9119 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:07.261068  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:07.289800  9271 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:07.290030  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.291184  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.232964 12952 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.165923 13021 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:07.293157  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.265115  9263 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:07.289124 12800 tablet_copy_client.cc:806] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
W20260709 13:05:07.293939  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.294396 12800 tablet_copy_client.cc:670] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:05:07.264717  9118 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:07.295783  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:07.296033  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.296402  9859 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:07.297137  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.298159  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.298759  9118 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:07.299422  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.288786  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 641) took 3416 ms. Trace:
W20260709 13:05:07.271674  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.287855  9122 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:07.251507 13114 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.303078  9115 rpcz_store.cc:276] 0709 13:05:03.872311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:04.566926 (+694615us) service_pool.cc:224] Handling call
0709 13:05:04.567157 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.035273 (+1468116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.288473 (+1253200us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54003863d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:07.288716 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3282860800}
I20260709 13:05:07.303951 12986 raft_consensus.cc:3055] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:07.326051 12986 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:07.326609 12977 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:07.327121 12986 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.328279 12986 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:07.304356  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.332700  9120 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:05:07.299685  9114 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
W20260709 13:05:07.338604 13097 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.466s	user 0.002s	sys 0.000s
I20260709 13:05:07.340797  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 993) took 1011 ms. Trace:
I20260709 13:05:07.314965  9271 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:05:07.345379  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.273392  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.315006  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:07.357633  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1922) took 1279 ms (client timeout 1689 ms). Trace:
I20260709 13:05:07.357797 12798 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:07.358048  9267 rpcz_store.cc:269] 0709 13:05:06.078332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.314676 (+1236344us) service_pool.cc:224] Handling call
0709 13:05:07.357573 (+ 42897us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:07.310951 12850 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:07.358887 13041 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.359048 12886 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:07.359998  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:07.360201 12886 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.360766  9858 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:07.361982  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:07.362540  9859 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:07.348649  9853 rpcz_store.cc:276] 0709 13:05:06.329161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.355354 (+ 26193us) service_pool.cc:224] Handling call
0709 13:05:07.340523 (+985169us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54005a87d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:07.340689 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1084906368}
I20260709 13:05:07.365264  9116 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:07.366552  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.347429  9122 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:07.302415  9121 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:07.176676 12966 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
W20260709 13:05:07.374464  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.380210  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 133799us
I20260709 13:05:07.381305  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:07.385315 12963 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:07.385797 12963 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:07.389606  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 198792us
I20260709 13:05:07.391705  9114 raft_consensus.cc:2376] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8. Re-sending same reply.
I20260709 13:05:07.414683  9859 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:07.430372  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:07.432001  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.432364  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.434525  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 690) took 1103 ms. Trace:
I20260709 13:05:07.435048  9197 rpcz_store.cc:276] 0709 13:05:06.331019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.331726 (+   707us) service_pool.cc:224] Handling call
0709 13:05:07.434282 (+1102556us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400398354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:07.434451 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940222208}
I20260709 13:05:07.438820 13011 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:07.456278  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:07.457139  9861 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:07.307700 13114 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:07.288414 12989 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:07.464289  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.468237  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:07.386258 12963 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:07.476647  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 46369.8us
I20260709 13:05:07.479496  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:07.480525  9116 raft_consensus.cc:2368] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11.
I20260709 13:05:07.429332 12953 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:07.485041  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.487668  9114 raft_consensus.cc:2368] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 6. Current term is 7.
I20260709 13:05:07.493260  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.496340  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.497195 12386 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:07.497982  9118 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:07.499012  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.499225  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.499571  9119 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:07.499734  9115 raft_consensus.cc:2376] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11. Re-sending same reply.
I20260709 13:05:07.502312 12979 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.499006  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.496044 12638 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:07.505275  9120 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:07.511332  9858 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:07.511456  9119 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:07.515062  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.519963  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.520650  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:07.521284  9266 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:07.521983  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:07.522368  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:07.523124  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.520617  9121 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:07.525364  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1920) took 1448 ms. Trace:
I20260709 13:05:07.494029 12990 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:07.534606  9861 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:05:07.535527  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:07.541318  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.542084  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.543488  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:07.525765  9269 rpcz_store.cc:276] 0709 13:05:06.077447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.115136 (+ 37689us) service_pool.cc:224] Handling call
0709 13:05:06.132114 (+ 16978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.137703 (+  5589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:06.137724 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:06.142472 (+  4748us) log.cc:844] Serialized 26 byte log entry
0709 13:05:06.266112 (+123640us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:06.266178 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:06.266295 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:07.337909 (+1071614us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:07.516396 (+178487us) raft_consensus.cc:1685] finished
0709 13:05:07.516423 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:07.525298 (+  8875us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1046,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":972,"spinlock_wait_cycles":28758272,"thread_start_us":123205,"threads_started":1,"wal-append.queue_time_us":729306}
I20260709 13:05:07.557976  9857 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:07.558256  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:07.561273  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:07.561734  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:07.567641  9116 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:07.571007  9861 raft_consensus.cc:2410] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:07.505225 12989 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:07.573609  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.574244  9271 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:07.577198  9122 raft_consensus.cc:2368] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8.
I20260709 13:05:07.573232  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.573120  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:07.580688  9193 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:07.595196  9268 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:05:07.611704  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 98908.8us
I20260709 13:05:07.574417  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.555347 13013 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:07.616968  9825 leader_election.cc:336] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.693s (SENT)
I20260709 13:05:07.621995  9120 raft_consensus.cc:1217] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Deduplicated request from leader. Original: 6.8->[8.9-8.9]   Dedup: 8.9->[]
I20260709 13:05:07.622649  9120 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:05:07.448206 13111 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.619506  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.638190  9116 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:07.634241  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:07.640053  9861 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:07.640573  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.641280  9118 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:07.565339 13114 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:07.574380 13121 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:07.655529  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.657589  9122 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:07.659508  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:07.659861 13002 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:07.663841  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.664480  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:07.655032  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:07.666108 12992 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:07.666723 12992 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:07.667045 12992 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:07.636965 12986 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:07.667444 12992 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:07.673549  9190 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:07.673599  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.674052  9191 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:07.674647  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.674983  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.675252  9191 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:07.675633  9198 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:07.676255  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:07.676991  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.678069  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 44890us
I20260709 13:05:07.676280  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.667555 12986 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.676611  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:07.676753  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.659508  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.672459  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.663506  9122 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:07.665998  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:07.680392  9198 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:07.688268  9266 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:07.689265  9853 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:07.692700  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.694517  9861 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:05:07.707783  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 90838.8us
I20260709 13:05:07.722091  9853 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:07.700829 13002 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:07.755173  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 89262.4us
W20260709 13:05:07.800601  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.830976  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.688885 13114 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:07.678267 12953 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:08.097767  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:07.956343  9856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.009s	sys 0.017s
W20260709 13:05:08.106594  9856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.014s	sys 0.026s
W20260709 13:05:08.021912  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 303347us
I20260709 13:05:08.098829  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:08.024993  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 314336us
I20260709 13:05:08.115535  9189 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:07.841683  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:07.986310  9858 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:05:08.005041  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:07.831635  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:08.121832  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.122550  9197 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:07.851027  9198 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:07.831282  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:07.837492  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.130620  9859 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:08.131227  9825 leader_election.cc:336] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 2.159s (SENT)
I20260709 13:05:07.836872  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:08.092039 12834 consensus_meta.cc:287] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.347s	user 0.008s	sys 0.006s
I20260709 13:05:08.130714  9118 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:08.135998  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:08.149276 12986 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:08.152909 13123 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.604s	user 0.000s	sys 0.001s
W20260709 13:05:08.160058  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.160429 12979 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:08.161146 12979 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:08.162349 12979 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:08.149106  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:08.167897  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2480) took 1100 ms. Trace:
I20260709 13:05:08.168752  9270 rpcz_store.cc:276] 0709 13:05:07.067854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.068015 (+   161us) service_pool.cc:224] Handling call
0709 13:05:07.069843 (+  1828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.092244 (+ 22401us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:07.596897 (+504653us) log.cc:844] Serialized 14 byte log entry
0709 13:05:07.816159 (+219262us) log.cc:844] Serialized 14 byte log entry
0709 13:05:08.167142 (+350983us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:08.167261 (+   119us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:08.167392 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.167533 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.167779 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":562349,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":99,"raft.queue_time_us":329174,"raft.run_wall_time_us":119066,"spinlock_wait_cycles":29411328,"thread_start_us":903443,"threads_started":3}
I20260709 13:05:08.172047  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:08.176005  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 44805.1us
W20260709 13:05:08.176764  9825 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 2.184s (SENT)
I20260709 13:05:08.177177  9825 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:08.143811 12834 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:08.199275  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:08.193333 13016 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:08.188697 13073 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:08.193751 12989 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:08.220157  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:08.222343  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 86432.2us
I20260709 13:05:08.223503  9164 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:08.219697 12992 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.001s
I20260709 13:05:08.222522 12989 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.236747  9861 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:08.241490  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:08.229020 13073 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:08.231290 13000 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:08.246204  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:08.245168  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1112) took 2340 ms. Trace:
W20260709 13:05:08.233362  9164 consensus_peers.cc:597] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:05:08.251132  9854 rpcz_store.cc:276] 0709 13:05:05.879271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:05.903588 (+ 24317us) service_pool.cc:224] Handling call
0709 13:05:05.904035 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:06.001467 (+ 97432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:06.001511 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:06.011002 (+  9491us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:06.011095 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.218917 (+2207822us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b5400700154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:08.218965 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.219142 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184766208}
W20260709 13:05:08.256354  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 414775us
I20260709 13:05:08.263996  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:08.267937  9195 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 13:05:08.285338  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.255154 13142 raft_consensus.cc:2764] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:08.239343 12992 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:08.244446 12989 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:08.325970  9195 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:08.326427  9192 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:08.328131  9190 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:08.295548 13114 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:08.266400 13116 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:05:08.337002 12963 consensus_meta.cc:287] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.822s	user 0.010s	sys 0.002s
W20260709 13:05:08.330674 13128 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:05:08.339762 13100 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.487s	user 0.010s	sys 0.006s
W20260709 13:05:08.340445 13137 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.123s	user 0.001s	sys 0.001s
W20260709 13:05:08.340240 13140 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 13:05:08.337595 13139 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.102s	user 0.004s	sys 0.001s
W20260709 13:05:08.351027  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 95921us
W20260709 13:05:08.357301  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.357910  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.358801  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:08.353637 12983 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:08.359463  9123 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.343273 13073 raft_consensus.cc:697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:08.364248  9116 consensus_meta.cc:287] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.680s	user 0.009s	sys 0.001s
I20260709 13:05:08.364634  9116 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:08.351547 12834 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:08.368844  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.371887 13131 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.468s	user 0.003s	sys 0.000s
W20260709 13:05:08.372931 13133 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.146s	user 0.001s	sys 0.001s
I20260709 13:05:08.361645 13114 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:08.374511 13114 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:08.376189  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1139) took 1015 ms. Trace:
I20260709 13:05:08.376612  9860 rpcz_store.cc:276] 0709 13:05:07.361295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.363413 (+  2118us) service_pool.cc:224] Handling call
0709 13:05:07.363756 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.364124 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.364162 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:07.431187 (+ 67025us) log.cc:844] Serialized 26 byte log entry
0709 13:05:07.688535 (+257348us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:07.688571 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.688644 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.375967 (+687323us) raft_consensus.cc:1685] finished
0709 13:05:08.375998 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.376130 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":311,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":6204,"thread_start_us":191768,"threads_started":1,"wal-append.queue_time_us":657856}
I20260709 13:05:08.377875 12992 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:08.380626  9123 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:08.381129  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 658) took 1132 ms. Trace:
I20260709 13:05:08.381430  9123 rpcz_store.cc:276] 0709 13:05:07.251624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.358423 (+1106799us) service_pool.cc:224] Handling call
0709 13:05:08.381075 (+ 22652us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:08.383452  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:08.384871  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:08.386121  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:08.386703  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:08.388533  9194 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 686) took 2059 ms (client timeout 1793 ms). Trace:
I20260709 13:05:08.350081 12963 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:08.388954  9194 rpcz_store.cc:269] 0709 13:05:06.329569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.384807 (+ 55238us) service_pool.cc:224] Handling call
0709 13:05:08.388303 (+2003496us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400397e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:08.388446 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766665984}
I20260709 13:05:08.390661 12963 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:08.392386 12989 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:08.393006 12989 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:08.393734 12989 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:08.393833  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:08.394129 12989 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:08.397913 12989 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:08.401131  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:08.402819  9117 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:08.405428  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1096) took 1110 ms. Trace:
I20260709 13:05:08.405849  9857 rpcz_store.cc:276] 0709 13:05:07.294808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.294956 (+   148us) service_pool.cc:224] Handling call
0709 13:05:07.295226 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:07.558432 (+263206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:08.405194 (+846762us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54007382d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:08.405367 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603855616}
I20260709 13:05:08.391325 12963 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:08.426923 13114 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:08.427618 13151 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:05:08.428373 13151 consensus_queue.cc:1243] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:08.428884  9120 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:08.427793  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.427096  9193 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:08.439177  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:08.421573 13116 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:08.439931 13116 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:08.440310 13116 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.449517  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.450505  9115 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:08.451025  9115 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:08.452198  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:08.452714  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.455219 13073 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:08.452819  9123 raft_consensus.cc:2376] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11. Re-sending same reply.
W20260709 13:05:08.479148  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 642389us
I20260709 13:05:08.459442  9115 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:08.421061 12963 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:08.486308  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.486807  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1137) took 1366 ms. Trace:
I20260709 13:05:08.487022  9120 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:05:08.489179  9115 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:08.489212  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.487190  9855 rpcz_store.cc:276] 0709 13:05:07.120108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.120255 (+   147us) service_pool.cc:224] Handling call
0709 13:05:07.137582 (+ 17327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:07.486807 (+349225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:07.486853 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:07.513399 (+ 26546us) log.cc:844] Serialized 26 byte log entry
0709 13:05:07.951389 (+437990us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:07.951451 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.951549 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:08.486615 (+535066us) raft_consensus.cc:1685] finished
0709 13:05:08.486634 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:08.486751 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5294,"raft.queue_time_us":39851,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"thread_start_us":626979,"threads_started":2,"wal-append.queue_time_us":663638}
I20260709 13:05:08.486083 12986 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:08.493575 12986 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:08.494048 12986 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:08.497159  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:08.502090  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:05:08.504297  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.504390  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 65276.2us
I20260709 13:05:08.505285  9161 leader_election.cc:400] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8.
I20260709 13:05:08.505623  9161 leader_election.cc:403] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:05:08.422112 12989 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:08.514276  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48326.9us
I20260709 13:05:08.486392 12963 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:08.517127  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.520442  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.521123  9116 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:08.522742  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:08.523895  9116 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:08.524056  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:08.518278  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:08.526371  9117 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:08.523729  9238 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 685fa3ec557f4f3db02ec6ccce1d6f3e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:08.531112  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.531508  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:08.528291  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:08.509212 12952 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:08.538256  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 129611us
I20260709 13:05:08.547394  9198 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:08.547322  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.525180 12834 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8.
I20260709 13:05:08.568360  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:08.575098  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:08.575927  9120 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:08.575178  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:08.534706 13002 raft_consensus.cc:2764] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:05:08.539392 13011 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:08.536970 12991 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:08.583950 13121 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:08.584098  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:08.584610 13121 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:08.584779  9117 raft_consensus.cc:2376] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8. Re-sending same reply.
I20260709 13:05:08.585072 12989 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:08.588606  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:08.589488  9194 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:08.596184  9825 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:08.540351 13111 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:08.598088  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.601755  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:08.605028  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.582297 13011 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:08.603724  9825 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:08.605767  9859 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:08.602756  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.606148  9856 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:08.607311  9853 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:08.595026 13008 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.575170 13157 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:05:08.624563  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.616329  9235 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:08.597069 12991 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:08.647217  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:08.666155  9123 pending_rounds.cc:85] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 8
I20260709 13:05:08.666594  9123 pending_rounds.cc:107] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:05:08.666826  9123 raft_consensus.cc:2889] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:08.606779 13095 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:05:08.625339  9862 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
W20260709 13:05:08.670173  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.684998  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 68719.1us
I20260709 13:05:08.689354  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:08.691176  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 71197us
I20260709 13:05:08.614687 13011 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:08.697665  9235 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:08.697914  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.618709 13126 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:08.700784  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 98869us
I20260709 13:05:08.704751  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:08.702521  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.596962 13114 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:05:08.586827 13121 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:08.745083  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.661468 13111 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:08.626140 13109 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.756747  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 159491us
I20260709 13:05:08.760686 13095 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:08.688865 12992 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:08.608445 13108 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 13:05:08.702605 13021 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.702632 13016 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:08.810595  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 105812us
I20260709 13:05:08.850452  9235 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:08.685520 13122 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:05:08.717109 13002 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:08.812433  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.624809 13166 consensus_queue.cc:1059] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.152s
I20260709 13:05:08.668974 13008 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:08.625591 13153 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.183s
I20260709 13:05:08.815855  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.700274 13169 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.131s
I20260709 13:05:08.816571  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.728742 13120 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:08.756850 13111 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:08.762338 12989 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:08.837702  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:08.853049  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 148366us
W20260709 13:05:08.848667  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 151269us
W20260709 13:05:08.865327  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.815915  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.838035 13013 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 13:05:08.884138  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.884150  9089 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:08.890152  9862 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:08.920193  9121 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:08.919972  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:08.881420 13111 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:08.881063 12992 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:08.932595  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 424655us
I20260709 13:05:08.863291 13095 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:05:08.941170  9162 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:08.865434 13021 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:08.952872  9238 connection.cc:256] RPC call timeout handler was delayed by -1.0567469240000946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
W20260709 13:05:08.956364  9238 connection.cc:256] RPC call timeout handler was delayed by -1.0090469240000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
I20260709 13:05:08.960520  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:08.917574 12989 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:08.964241 13165 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
W20260709 13:05:08.964681 13158 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.429s	user 0.002s	sys 0.002s
I20260709 13:05:08.920208 13120 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:08.971271 12989 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:08.971866 12989 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:08.972121 13164 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 13:05:08.972365 12989 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:08.974368  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 110405us
I20260709 13:05:08.930646 13016 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:08.976058 13016 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:08.978274 13016 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:08.975244  9235 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:08.964407 13167 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 13:05:08.982832  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 217375us
I20260709 13:05:08.974529 12989 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:08.848713 12991 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:08.992353  9115 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
W20260709 13:05:08.958472 13163 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.354s	user 0.001s	sys 0.000s
I20260709 13:05:08.938493 13002 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:05:09.001768  9087 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.737s (SENT)
I20260709 13:05:09.004230  9115 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:05:08.955076 13161 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.426s	user 0.000s	sys 0.001s
I20260709 13:05:08.976065 13171 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:08.956334 13114 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:08.868232 13109 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:09.116107  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 98726.9us
I20260709 13:05:09.121580  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:09.142810  9272 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:09.142982  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.124118  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.143685  9266 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:09.126498  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:09.064251  9118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.000s	sys 0.008s
W20260709 13:05:09.144476  9118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.005s	sys 0.013s
I20260709 13:05:09.127405  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.127858  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.006901  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:09.136183  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.145800  9121 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:09.007189  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:08.956199 13166 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:09.146960  9270 raft_consensus.cc:2368] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14.
I20260709 13:05:09.145877  9862 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:09.129161  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2100) took 1454 ms. Trace:
I20260709 13:05:09.132958  9191 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:09.126957  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:08.956176 12833 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:09.151495  9271 raft_consensus.cc:2410] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:09.150202  9190 rpcz_store.cc:276] 0709 13:05:07.674646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.677681 (+  3035us) service_pool.cc:224] Handling call
0709 13:05:07.679007 (+  1326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.330606 (+651599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.128857 (+798251us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b540010cf54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:09.129098 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1472524672}
I20260709 13:05:09.154971  9264 raft_consensus.cc:2368] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6.
I20260709 13:05:09.162622  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.163601  9270 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:09.164749  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:09.125844 13116 consensus_meta.cc:287] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.000s
I20260709 13:05:09.167627  9269 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
W20260709 13:05:09.106823 12986 consensus_meta.cc:287] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.612s	user 0.006s	sys 0.004s
W20260709 13:05:09.133816 13127 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.029s	user 0.003s	sys 0.000s
W20260709 13:05:09.169159  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 213981us
W20260709 13:05:09.106528 13175 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.053s	user 0.000s	sys 0.003s
I20260709 13:05:09.052244 13126 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:09.121969 13008 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:09.007012 13173 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:09.072925 13168 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:09.240535  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.228586  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:09.210753  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.228257  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:09.236164  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 352085us
I20260709 13:05:09.279366  9198 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:09.240816  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 97778.5us
I20260709 13:05:09.072865 12992 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:09.252889  9862 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:09.220347  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.229346  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.188931  9194 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:09.226230  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.284584  9266 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:09.248260  9272 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
W20260709 13:05:09.272272  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 270541us
I20260709 13:05:09.121062 13021 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:09.149739 13166 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:09.289357  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.290494  9272 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:05:09.291039  9272 pending_rounds.cc:85] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 8
I20260709 13:05:09.291294  9272 pending_rounds.cc:107] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:05:09.145409 13171 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:09.152055 13002 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:09.210919  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:09.224964 12963 consensus_meta.cc:287] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.000s
I20260709 13:05:09.292428  9271 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.155953 12833 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:09.297096  9267 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:09.279614  9196 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:09.279953  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:09.188479 12893 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.211217 13111 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.162313 13114 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.277458 13168 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:09.287339  9162 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:09.229835  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.316886  9853 raft_consensus.cc:2393] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:09.300267  9087 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.236s (SENT)
I20260709 13:05:09.317800 13002 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:09.324723  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:09.325024 13126 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.210275 13116 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.305431  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:09.293807  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.271374 12986 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:09.309914  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.312541  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:09.293511 13166 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:09.303706 13199 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:05:09.299814 12833 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:09.347580  9269 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:09.348724  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.349526  9269 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:09.338024 13021 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.350524 13021 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.352939 13021 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:09.316169  9198 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:09.297724  9235 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:09.291635  9272 raft_consensus.cc:2889] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:09.332470  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1921) took 3254 ms. Trace:
I20260709 13:05:09.326160  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:09.327209 13011 consensus_meta.cc:287] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.597s	user 0.003s	sys 0.008s
I20260709 13:05:09.300405 12963 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.592916 13011 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:09.331816  9087 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:09.594421 12963 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:09.594980  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 299581us
I20260709 13:05:09.340751  9271 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:09.309612 13173 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:09.332862 13168 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.343631 12986 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:09.324753 13109 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:09.304708 12893 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:09.461761  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.354476  9193 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:09.356326  9196 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:09.438393  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.603497  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.604043  9189 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:09.604699  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.604894  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.605365  9196 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:09.437903  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:09.484241  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 196968us
W20260709 13:05:09.489943  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 192287us
I20260709 13:05:09.603531 12893 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:09.607915 13201 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:09.517789  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.518925  9265 rpcz_store.cc:276] 0709 13:05:06.077894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:06.078158 (+   264us) service_pool.cc:224] Handling call
0709 13:05:06.078631 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:06.079316 (+   685us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:06.132487 (+ 53171us) log.cc:844] Serialized 14 byte log entry
0709 13:05:07.283105 (+1150618us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b1800392380. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:05:07.511948 (+228843us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:07.556270 (+ 44322us) log.cc:844] Serialized 26 byte log entry
0709 13:05:07.556435 (+   165us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:07.556475 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:07.556574 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.131680 (+1575106us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:09.332206 (+200526us) raft_consensus.cc:1685] finished
0709 13:05:09.332240 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.332405 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4814,"prepare.queue_time_us":72582,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1809,"raft.queue_time_us":5530,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":28678,"spinlock_wait_cycles":29759616,"thread_start_us":280653,"threads_started":2,"wal-append.queue_time_us":1601782}
W20260709 13:05:09.465862  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 153404us
I20260709 13:05:09.437990  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2300) took 1078 ms. Trace:
W20260709 13:05:09.589082  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 66513.5us
I20260709 13:05:09.611385  9089 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:09.313710 13121 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.612255  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:09.612545 13121 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.613474 13180 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:09.613860 13180 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:09.614207 13180 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:09.614733 13121 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:09.369458 13195 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:09.615671 13195 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:09.616225 13195 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:09.445075 12869 tablet_bootstrap.cc:492] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:09.595350  9263 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:09.315035 13008 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:05:09.595942  9197 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:09.593896  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:09.605937  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.617928  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:09.610899  9118 rpcz_store.cc:276] 0709 13:05:08.359236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.360876 (+  1640us) service_pool.cc:224] Handling call
0709 13:05:08.361100 (+   224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:08.495979 (+134879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.496018 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:08.496512 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:05:09.144997 (+648485us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:09.187066 (+ 42069us) log.cc:844] Serialized 14 byte log entry
0709 13:05:09.189287 (+  2221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.189385 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.350850 (+161465us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b1800499d20. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:09.356244 (+  5394us) raft_consensus.cc:1685] finished
0709 13:05:09.437598 (+ 81354us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.437783 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124804,"prepare.queue_time_us":41061,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":172590,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":14104,"spinlock_wait_cycles":34924288,"thread_start_us":648097,"threads_started":1,"wal-append.queue_time_us":729850}
I20260709 13:05:09.632742  9272 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:09.582571 13157 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:09.637367  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.640480  9266 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:09.639516  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.581643 12992 raft_consensus.cc:2764] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:05:09.599009 12807 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:09.651182  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:09.643317 13202 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.652345  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.652674 13202 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:09.655225 12869 tablet_bootstrap.cc:492] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:05:09.656129 12869 ts_tablet_manager.cc:1403] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 5.090s	user 0.052s	sys 0.057s
I20260709 13:05:09.659945  9272 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:05:09.640700 12963 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:09.665895  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.664002 13109 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.667104 13109 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.668187  9272 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:09.669425 13109 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:09.670974  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.640869 13191 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.684628  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1099) took 2220 ms. Trace:
I20260709 13:05:09.661819 13116 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:09.664422 12963 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.685277  9861 rpcz_store.cc:276] 0709 13:05:07.463863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:07.715053 (+251190us) service_pool.cc:224] Handling call
0709 13:05:08.234004 (+518951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:08.239153 (+  5149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.684343 (+1445190us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54006216d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:09.684546 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574641024}
I20260709 13:05:09.692149  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.617136 13195 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.703239  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:09.704211 13111 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:09.706918 13159 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.138s	user 0.000s	sys 0.007s
I20260709 13:05:09.327224 13171 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:09.731204  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 124141us
W20260709 13:05:09.731946  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 119759us
I20260709 13:05:09.695355 12963 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:09.694548 13126 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:09.743848  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:09.731945 13204 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:05:09.725045 13114 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:09.750612  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.733222  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:09.737046  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.761235  9162 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:09.754122  9267 raft_consensus.cc:2376] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8. Re-sending same reply.
I20260709 13:05:09.767835  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2306) took 1133 ms. Trace:
I20260709 13:05:09.768040  9828 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:09.734896 13173 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:09.768388  9123 rpcz_store.cc:276] 0709 13:05:08.634604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.634775 (+   171us) service_pool.cc:224] Handling call
0709 13:05:08.635272 (+   497us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:08.689211 (+ 53939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:08.689261 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:08.716718 (+ 27457us) log.cc:844] Serialized 52 byte log entry
0709 13:05:09.275661 (+558943us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:09.301529 (+ 25868us) log.cc:844] Serialized 14 byte log entry
0709 13:05:09.306827 (+  5298us) log.cc:844] Serialized 14 byte log entry
0709 13:05:09.310994 (+  4167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:09.311067 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:09.767534 (+456467us) raft_consensus.cc:1685] finished
0709 13:05:09.767560 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:09.767734 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50736,"prepare.queue_time_us":26208,"prepare.run_wall_time_us":37,"raft.queue_time_us":276837,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":26793,"spinlock_wait_cycles":99356416,"thread_start_us":389931,"threads_started":1,"wal-append.queue_time_us":547836}
I20260709 13:05:09.734984 12893 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:05:09.764483 12966 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:09.790971  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.791705  9267 raft_consensus.cc:2368] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6.
I20260709 13:05:09.792622  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:09.793277  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:09.793591  9118 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:09.794067  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:09.755043 13111 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:05:09.801452  9118 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:05:09.800249  9271 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:09.802599  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:09.803061  9118 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:09.764164 13207 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:05:09.804239  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:09.784333 13116 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:09.833069  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:09.745877 13126 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.761957 13166 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.810155  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:09.764366 13021 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:05:09.792649 12966 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:09.812614 13203 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.316s	user 0.001s	sys 0.001s
I20260709 13:05:09.790568 12869 raft_consensus.cc:359] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:09.816433 13173 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:09.805874 13210 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:05:09.820412  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:09.853600  9120 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:09.820984 13209 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.821619  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:09.856545  9122 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:05:09.822520  9266 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:09.792095 13109 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:09.823426  9119 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:05:09.860139  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.860956  9266 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:09.861796  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.862883  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:09.823972 13205 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:05:09.824466  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.865422  9267 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:09.865491  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.866405  9268 raft_consensus.cc:2410] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:09.866330  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:09.867177  9267 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:09.867319  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:09.826710  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:09.828333 13192 raft_consensus.cc:3055] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:09.872264  9268 raft_consensus.cc:2368] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14.
W20260709 13:05:09.832017  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 64058.5us
I20260709 13:05:09.688741 13191 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:09.836725  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.873780  9272 raft_consensus.cc:2410] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:05:09.833777  9117 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
W20260709 13:05:09.838740  9826 leader_election.cc:336] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.573s (SENT)
I20260709 13:05:09.874850  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:09.836270  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:09.850618  9115 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:05:09.843245 12966 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:09.851397 12990 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:09.849525 12869 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:05:09.855554  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:09.877964  9271 raft_consensus.cc:2368] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6.
I20260709 13:05:09.878760  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.879469  9271 raft_consensus.cc:2368] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14.
I20260709 13:05:09.825114  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:09.880203  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:09.881259  9271 raft_consensus.cc:2376] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9. Re-sending same reply.
I20260709 13:05:09.882272  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:09.868916  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.868427  9266 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:09.883015  9271 raft_consensus.cc:2376] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8. Re-sending same reply.
I20260709 13:05:09.886690  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.887822  9268 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:05:09.888796  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52609.9us
I20260709 13:05:09.890116  9825 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:09.883376  9267 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
W20260709 13:05:09.894833  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 68218.4us
W20260709 13:05:09.895296  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 52515.1us
I20260709 13:05:09.904724  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.876294 13192 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:09.908435  9271 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:09.877394 13210 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.909132  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:09.911196  9235 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:09.914844  9162 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:09.913942  9266 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:05:09.917827  9267 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:05:09.893028 13146 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:09.930681  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:09.981578  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 96245.9us
I20260709 13:05:09.910867 13021 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:09.986230  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:09.988534  9271 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:09.994414  9266 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:09.996475  9267 raft_consensus.cc:2376] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6. Re-sending same reply.
I20260709 13:05:09.914248 13116 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:09.915717 13214 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:09.910157 12992 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:05:09.921525 13114 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:09.983135 13186 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:09.916191 13109 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:09.881557 13209 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:10.047570  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:10.039359  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 122279us
I20260709 13:05:10.048890  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:10.048959  9826 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.652s (SENT)
I20260709 13:05:09.984040 12991 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:09.980811 13195 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.920418 13218 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 13:05:10.047303 13109 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.048063 13021 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.991242 13210 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:10.081630  9116 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:05:10.082536  9118 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
W20260709 13:05:10.089178  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 189025us
I20260709 13:05:10.067093 12991 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:09.922191 12869 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:10.068048 13114 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:10.084540 13225 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.086475 13109 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:10.042145 13214 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.094265 13146 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:05:10.082583 13195 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.042181 13192 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:10.158058  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 109150us
W20260709 13:05:10.170642  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.167970 13191 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:10.207573  9826 leader_election.cc:336] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.666s (SENT)
I20260709 13:05:10.208138  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:10.188505  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.214466  9271 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:10.094148 13126 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:10.125442 12833 consensus_meta.cc:287] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.777s	user 0.010s	sys 0.002s
I20260709 13:05:10.219990 13114 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.222553 13220 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:10.223649 13169 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.111481 12992 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.104664 12991 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:10.112825 13207 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:10.111343 13166 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:10.089774 13021 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.242862  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:10.246817  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:10.248072  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 331937us
W20260709 13:05:10.250741  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.252139  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.253707  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.262224  9825 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:10.238214 13192 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:10.303117  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:10.305750  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43616.7us
I20260709 13:05:10.306500  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:10.307577  9825 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:10.312639  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 230696us
I20260709 13:05:10.314249  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:10.314495  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.320248  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:10.250200 13169 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.255393 13126 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:10.331378  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 123847us
I20260709 13:05:10.333966  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:10.334283  9826 leader_election.cc:336] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.774s (SENT)
I20260709 13:05:10.335958  9856 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:10.256582 13207 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:10.113355 13225 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.341367  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:10.241459 12833 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.368875  9855 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:10.371769  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.316560 13166 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:10.256197 12992 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:10.375519  9826 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.783s (SENT)
I20260709 13:05:10.325726 13169 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:10.305442 13220 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:05:10.386942  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.131170 13116 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.385533  9826 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:10.328287 13227 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.390847  9198 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:10.434355  9271 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:10.422232  9266 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:05:10.445688  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:10.455116  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 79618.1us
W20260709 13:05:10.455925  9826 leader_election.cc:336] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.793s (SENT)
I20260709 13:05:10.456329  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:10.365442 13126 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.389492 13231 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:10.460873  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 142810us
I20260709 13:05:10.424352 13229 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:10.459472 12966 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.000s
I20260709 13:05:10.509944 12966 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.511155 13209 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:10.391403 13166 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.512030 13214 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:10.512609 13157 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:10.513101 13210 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:10.513576 13166 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:10.514698 13157 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:10.515152 13157 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:10.517656 13157 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.518985 13157 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:10.438647 13227 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.522506 13227 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:10.524329  9196 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:10.486996  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:10.527503 12963 consensus_meta.cc:287] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.781s	user 0.010s	sys 0.001s
I20260709 13:05:10.493330 12869 ts_tablet_manager.cc:1434] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.837s	user 0.019s	sys 0.015s
I20260709 13:05:10.494737 12992 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:10.496234 13149 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 2.154s	user 0.010s	sys 0.011s
I20260709 13:05:10.451825  9266 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:05:10.533084  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:10.499161  9862 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:05:10.497190 13229 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.461043 13235 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:10.502931 13180 consensus_meta.cc:287] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.888s	user 0.013s	sys 0.000s
I20260709 13:05:10.463891 13207 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.503607  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:10.551057  9825 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:10.552498  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:10.553210  9195 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
W20260709 13:05:10.504791 13228 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:05:10.505362 13126 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:10.507118 13232 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:10.509279  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.510317 13225 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:10.485248 13021 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:10.492678 13120 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:10.529788  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 73900.1us
I20260709 13:05:10.545794  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.570026  9862 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:10.544723 13235 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:10.536267 13231 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:10.574268  9267 raft_consensus.cc:2410] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:05:10.532835 12869 ts_tablet_manager.cc:1467] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Tablet startup took 875ms
I20260709 13:05:10.544474 13220 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:10.584288  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:10.544384 13229 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:10.544787 13233 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.578737  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.535270 12963 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:10.578787  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 108885us
W20260709 13:05:10.582384  9826 leader_election.cc:336] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.874s (SENT)
W20260709 13:05:10.597056  9197 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2106) took 1420 ms (client timeout 1521 ms). Trace:
I20260709 13:05:10.597270  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:10.561581 12992 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:10.597685  9197 rpcz_store.cc:269] 0709 13:05:09.176434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.176572 (+   138us) service_pool.cc:224] Handling call
0709 13:05:10.596764 (+1420192us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54001d4f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:10.596965 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731346176}
I20260709 13:05:10.600119  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.601277  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:10.604843  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:10.605533  9123 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:10.610138  9858 raft_consensus.cc:1217] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEARNER]: Deduplicated request from leader. Original: 6.5->[6.6-6.6]   Dedup: 6.6->[]
I20260709 13:05:10.610164  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:10.604638  9266 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:10.612921  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:10.577438 13243 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:10.563475 12910 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
W20260709 13:05:10.615989  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.616212  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.646909  9266 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:10.647774  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:10.648099  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.585573 13011 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:10.618675  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:10.581772 13235 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:10.588542 12966 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:10.654412  9122 raft_consensus.cc:2410] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:05:10.626523  9272 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:10.561318 13180 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:10.645797  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63446.8us
W20260709 13:05:10.671277  9826 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.939s (SENT)
I20260709 13:05:10.650435  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:10.671814  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:10.653568  9120 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:10.652854  9266 raft_consensus.cc:2410] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:10.690119  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.620218 13114 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:10.694306  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:10.694729  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.602200 13230 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:05:10.708209  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.708927  9266 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
W20260709 13:05:10.709419  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.709758  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.654299 13243 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:10.587633 12869 ts_tablet_manager.cc:1469] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Trace:
0709 13:05:00.072856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.075407 (+  2551us) service_pool.cc:224] Handling call
0709 13:05:00.617730 (+542323us) ts_tablet_manager.cc:934] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
0709 13:05:01.775521 (+1157791us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:05:02.231343 (+455822us) tablet_metadata.cc:665] Metadata flushed
0709 13:05:02.231468 (+   125us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:05:02.245601 (+ 14133us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:05:02.245916 (+   315us) tablet_replica.cc:206] Creating consensus instance
0709 13:05:02.246248 (+   332us) ts_tablet_manager.cc:989] Replica registered.
0709 13:05:02.246314 (+    66us) inbound_call.cc:177] Queueing success response
0709 13:05:04.566061 (+2319747us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:05:09.656483 (+5090422us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:05:09.701810 (+ 45327us) tablet_replica.cc:261] Starting instrumentation
0709 13:05:09.702415 (+   605us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":21938,"mutex_wait_us":1569,"raft.queue_time_us":109425,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"tablet-copy.queue_time_us":399015,"tablet-download-pool.queue_time_us":194,"tablet-download-pool.run_cpu_time_us":1254,"tablet-download-pool.run_wall_time_us":1092925,"thread_start_us":551205,"threads_started":3}
W20260709 13:05:10.721081 13247 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 13:05:10.726190  9121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 2305) took 2093 ms (client timeout 1727 ms). Trace:
I20260709 13:05:10.728302 13095 raft_consensus.cc:3055] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:10.779901 13095 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:10.780890 13095 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:10.702718 13221 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.538s	user 0.003s	sys 0.001s
I20260709 13:05:10.615963 12992 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.784117 12992 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:10.733862  9114 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 13:05:10.735911 13245 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:05:10.736035  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:10.784544 13243 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.785282  9859 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:10.740775 12963 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:10.724769 13171 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:10.744364 13109 consensus_meta.cc:287] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.613s	user 0.003s	sys 0.011s
I20260709 13:05:10.696884 13231 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:10.748028 13145 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:10.750018  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 701) took 2270 ms. Trace:
I20260709 13:05:10.791656  9191 rpcz_store.cc:276] 0709 13:05:08.480053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.480210 (+   157us) service_pool.cc:224] Handling call
0709 13:05:09.132528 (+652318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.150858 (+ 18330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.749724 (+1598866us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540010cf50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:10.749925 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463682816}
I20260709 13:05:10.752043  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:10.793149  9270 raft_consensus.cc:2368] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 5. Current term is 6.
W20260709 13:05:10.793578  9270 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 826) took 1582 ms (client timeout 1793 ms). Trace:
W20260709 13:05:10.793927  9270 rpcz_store.cc:269] 0709 13:05:09.210821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.223776 (+ 12955us) service_pool.cc:224] Handling call
0709 13:05:10.751899 (+1528123us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:05:10.793522 (+ 41623us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4807040}
W20260709 13:05:10.755519  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.758976  9861 raft_consensus.cc:2410] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:05:10.759617 12991 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:10.602596 13233 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.759702  9020 catalog_manager.cc:5697] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:10.761857  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:10.771639  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:10.777268  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 106037us
W20260709 13:05:10.798532  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.798627  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:10.799008  9826 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.600s (SENT)
I20260709 13:05:10.778518  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:10.778242  9121 rpcz_store.cc:269] 0709 13:05:08.632812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.632979 (+   167us) service_pool.cc:224] Handling call
0709 13:05:10.725901 (+2092922us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b540023bed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:10.726072 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528952576}
I20260709 13:05:10.799363  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:10.801891  9826 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.961s (SENT)
W20260709 13:05:10.803015  9826 leader_election.cc:336] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.002s (SENT)
I20260709 13:05:10.804683  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.805311  9270 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:10.808935  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.809950  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.810571  9270 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:05:10.811389  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.809579  9266 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:10.812353  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:10.812978  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:10.813671  9266 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:05:10.809894  9854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1004) took 2060 ms (client timeout 1606 ms). Trace:
W20260709 13:05:10.814785  9854 rpcz_store.cc:269] 0709 13:05:08.745695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.746266 (+   571us) service_pool.cc:224] Handling call
0709 13:05:10.761636 (+2015370us) spinlock_profiling.cc:243] Waited 983 us on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:05:10.805658 (+ 44022us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2261760}
I20260709 13:05:10.820513  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:10.821065  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:10.821151  9272 raft_consensus.cc:2376] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10. Re-sending same reply.
I20260709 13:05:10.821712  9121 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:10.830205  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.771765 13242 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.806344 12908 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:10.838507  9266 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:10.839671  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:10.840263  9272 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:10.843014  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:10.836920  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.846691  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.844734  9270 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:05:10.848496  9854 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:10.849735  9854 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:10.797312 13109 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.842342 13242 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.859565  9860 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:10.852737  9266 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:10.862738  9266 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
W20260709 13:05:10.863382  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:10.865687 13109 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:10.869285  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 61334.2us
I20260709 13:05:10.877962  9272 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:10.849759  9270 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:05:10.881158  9121 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:05:10.871331  9826 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.099s (SENT)
I20260709 13:05:10.879825  9270 pending_rounds.cc:85] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 4
I20260709 13:05:10.938903  9270 pending_rounds.cc:107] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:05:10.939159  9270 raft_consensus.cc:2889] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:10.939898  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.940529  9272 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:10.864228 12979 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:10.944840 13202 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:10.970134  9265 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:05:10.971125  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 111314us
W20260709 13:05:10.972482  9853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 1111) took 2330 ms (client timeout 2382 ms). Trace:
W20260709 13:05:10.973182  9853 rpcz_store.cc:269] 0709 13:05:08.617732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.661630 (+ 43898us) service_pool.cc:224] Handling call
0709 13:05:10.948180 (+2286550us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b54006e9e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:10.948348 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1414488960}
I20260709 13:05:10.961535  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 685) took 1164 ms. Trace:
I20260709 13:05:10.956012  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:10.965353  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:10.946480  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 686) took 1149 ms. Trace:
I20260709 13:05:11.812422 13265 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.952852 13265 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:11.992734  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:12.958463  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:11.757659  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0586013949998687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23820000000000002s for a response
W20260709 13:05:12.098371  9087 connection.cc:256] RPC call timeout handler was delayed by -1.1458836269999892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
W20260709 13:05:13.004025  9087 connection.cc:256] RPC call timeout handler was delayed by -1.141383626999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
W20260709 13:05:13.004428  9087 connection.cc:256] RPC call timeout handler was delayed by -1.1269836269999587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 13:05:13.004756  9087 connection.cc:256] RPC call timeout handler was delayed by -1.0963836270000229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
W20260709 13:05:13.005158  9087 connection.cc:256] RPC call timeout handler was delayed by -1.0675836269999763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 13:05:13.005767  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.506184  9115 rpcz_store.cc:276] 0709 13:05:09.796623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.796734 (+   111us) service_pool.cc:224] Handling call
0709 13:05:09.825833 (+ 29099us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.876864 (+ 51031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.961284 (+1084420us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b5400254c54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:10.961475 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56380160}
I20260709 13:05:11.385393  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.030997  9117 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:12.045461  9120 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
W20260709 13:05:13.036300  9120 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2315) took 2088 ms (client timeout 1567 ms). Trace:
W20260709 13:05:13.036757  9120 rpcz_store.cc:269] 0709 13:05:10.947578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.965097 (+ 17519us) service_pool.cc:224] Handling call
0709 13:05:13.036217 (+2071120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:11.568161  9121 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.041821  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.042587  9197 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:10.971939  9161 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:13.046682  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 2.0747e+06us
W20260709 13:05:11.394428  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.050035  9087 leader_election.cc:336] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.500s (SENT)
I20260709 13:05:13.050607  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:12.094743  9189 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.138s	user 0.002s	sys 0.014s
W20260709 13:05:13.054143  9189 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.098s	user 0.004s	sys 0.016s
I20260709 13:05:11.847138  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 702) took 3347 ms. Trace:
I20260709 13:05:12.049057  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 720) took 1590 ms. Trace:
I20260709 13:05:11.989109  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 717) took 1684 ms. Trace:
I20260709 13:05:13.056511  9196 rpcz_store.cc:276] 0709 13:05:10.458540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.458932 (+   392us) service_pool.cc:224] Handling call
0709 13:05:10.498002 (+ 39070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.524764 (+ 26762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.048760 (+1523996us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b5400217d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:12.048985 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60087552}
I20260709 13:05:13.057232  9198 rpcz_store.cc:276] 0709 13:05:10.304595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.304720 (+   125us) service_pool.cc:224] Handling call
0709 13:05:10.318935 (+ 14215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.433082 (+114147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.988785 (+1555703us) spinlock_profiling.cc:243] Waited 71.6 ms on lock 0x7b54000677d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:11.989037 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164743552}
I20260709 13:05:13.056054  9193 rpcz_store.cc:276] 0709 13:05:08.499521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:08.499716 (+   195us) service_pool.cc:224] Handling call
0709 13:05:08.500058 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:09.602773 (+1102715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.846879 (+2244106us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b5400434ed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:11.847072 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1964600064}
W20260709 13:05:11.829742  9235 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.709s (SENT) [suppressed 55 similar messages]
W20260709 13:05:11.726972  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:11.570510  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:13.061939  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 1.23215e+06us
W20260709 13:05:11.745886  9263 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 820) took 2623 ms (client timeout 1666 ms). Trace:
I20260709 13:05:11.691859  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:10.979076  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:13.065750  9265 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
W20260709 13:05:13.066375  9265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1962) took 2096 ms (client timeout 2238 ms). Trace:
W20260709 13:05:13.066778  9265 rpcz_store.cc:269] 0709 13:05:10.970318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.972329 (+  2011us) service_pool.cc:224] Handling call
0709 13:05:13.066306 (+2093977us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.067793  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:13.068202  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 73039.2us
I20260709 13:05:13.068339  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2309) took 2314 ms. Trace:
W20260709 13:05:11.613493  9266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.006s	sys 0.009s
I20260709 13:05:11.900295  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2508) took 2099 ms. Trace:
I20260709 13:05:11.735283  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:10.950261  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:13.075944  9270 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:05:13.074487  9268 rpcz_store.cc:276] 0709 13:05:09.800867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.901910 (+101043us) service_pool.cc:224] Handling call
0709 13:05:09.902262 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:09.906752 (+  4490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:09.906814 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:09.907430 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 13:05:10.057977 (+150547us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:10.206312 (+148335us) log.cc:844] Serialized 14 byte log entry
0709 13:05:10.500335 (+294023us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b18002ed4a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:11.003025 (+502690us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b18002ed4a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287 00007f76566ff208
0709 13:05:11.329413 (+326388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:11.329538 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:11.898222 (+568684us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b18002ed4a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:11.900070 (+  1848us) raft_consensus.cc:1685] finished
0709 13:05:11.900089 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:11.900222 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":138407,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":10359,"raft.queue_time_us":10260,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":5262,"spinlock_wait_cycles":26098048,"thread_start_us":453601,"threads_started":2,"wal-append.queue_time_us":327465,"wal-append.run_cpu_time_us":7256,"wal-append.run_wall_time_us":1068103}
W20260709 13:05:13.076411  9270 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1961) took 2127 ms (client timeout 1547 ms). Trace:
W20260709 13:05:13.076768  9270 rpcz_store.cc:269] 0709 13:05:10.948978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.950032 (+  1054us) service_pool.cc:224] Handling call
0709 13:05:13.076352 (+2126320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.076853  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:13.077538  9268 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.077740  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:13.078269  9270 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:11.864251  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 866) took 1561 ms. Trace:
I20260709 13:05:13.080399  9271 rpcz_store.cc:276] 0709 13:05:10.302457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.302764 (+   307us) service_pool.cc:224] Handling call
0709 13:05:11.864035 (+1561271us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54000fafd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:11.864175 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299630464}
I20260709 13:05:13.081866  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:12.807543  9009 maintenance_manager.cc:643] P cdc5c504a10a4da1812e571d14ae3abd: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 10.589s	user 0.169s	sys 0.056s Metrics: {"bytes_written":29163,"cfile_init":1,"dirs.queue_time_us":11981,"dirs.run_cpu_time_us":466,"dirs.run_wall_time_us":7456,"drs_written":1,"lbm_read_time_us":370,"lbm_reads_lt_1ms":4,"lbm_write_time_us":160174,"lbm_writes_10-100_ms":4,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":63,"thread_start_us":11977,"threads_started":2,"wal-append.queue_time_us":783}
W20260709 13:05:12.818953  9119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.937s	user 0.009s	sys 0.014s
I20260709 13:05:12.837545  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:12.848047 13269 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:12.858824  9272 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.009s	sys 0.006s
W20260709 13:05:12.866307  9116 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2312) took 3070 ms (client timeout 1709 ms). Trace:
I20260709 13:05:12.882794  9020 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 9, now has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 9 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:05:12.895397  9018 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36634 (request call id 340) took 26119 ms (client timeout 15000 ms). Trace:
I20260709 13:05:12.773440  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1153) took 2456 ms. Trace:
I20260709 13:05:12.860785  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:12.185436 12953 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:10.966800 12979 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:12.839115 13156 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.110940  9270 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:11.821902 13195 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:12.802436 13095 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:11.761055  9825 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:12.067036 13184 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:05:10.979067  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 107755us
W20260709 13:05:13.150377  9826 leader_election.cc:336] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.100s (SENT)
W20260709 13:05:12.097030  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:12.020490  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 1114) took 1787 ms. Trace:
W20260709 13:05:12.776695  9883 debug-util.cc:398] Leaking SignalData structure 0x7b08002648a0 after lost signal to thread 8024
I20260709 13:05:11.786442 12833 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:10.957676 12991 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:12.039319 13151 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:10.957762 13152 consensus_peers.cc:313] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Unable to generate Tablet Copy request for peer: Not found: Peer not tracked or queue not in leader mode.
I20260709 13:05:12.012929 13168 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:11.319967 13171 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.533s	user 0.012s	sys 0.000s
W20260709 13:05:10.954038 13178 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 2.027s	user 0.001s	sys 0.011s
I20260709 13:05:11.561823 13182 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:11.873831 13207 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:12.650996 13215 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 2.527s	user 0.007s	sys 0.000s
I20260709 13:05:10.962925 13229 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.951120 13230 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:12.097203 13231 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:10.942039 13232 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:10.968724 13233 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:11.002960 13235 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:10.985001 13242 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:11.431223 13248 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.881s	user 0.003s	sys 0.003s
I20260709 13:05:12.889986 13180 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:11.761222 13263 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:12.931356  9219 debug-util.cc:398] Leaking SignalData structure 0x7b0800168c20 after lost signal to thread 8024
I20260709 13:05:12.952589  9114 rpcz_store.cc:276] 0709 13:05:09.796949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.797061 (+   112us) service_pool.cc:224] Handling call
0709 13:05:10.733534 (+936473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:10.784811 (+ 51277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:10.946162 (+161351us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b540023bed0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:10.946292 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2100605312}
I20260709 13:05:12.955602 13265 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:12.988701  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:12.990262  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2308) took 2236 ms. Trace:
I20260709 13:05:13.001483 13166 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.001693  9207 tablet_copy_service.cc:342] P 94504d29c6744f38bb06943479df9466: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-1c199cae5ff14a478726c8a8b491f8f7 received from {username='slave'} at 127.0.0.1:56646
I20260709 13:05:13.002926  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.015259 13111 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:13.159686  9861 rpcz_store.cc:276] 0709 13:05:10.232989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.233280 (+   291us) service_pool.cc:224] Handling call
0709 13:05:12.020233 (+1786953us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54004538d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:12.020413 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1150883584}
I20260709 13:05:13.038259 13275 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.203058 13275 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.205433 13275 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:12.010093  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.045243 13274 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:13.048898 13276 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.207569 13274 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.208123 13276 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:13.162688  9883 debug-util.cc:398] Leaking SignalData structure 0x7b08002649e0 after lost signal to thread 9055
I20260709 13:05:13.210022 13274 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:13.210218 13276 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.051393 13201 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:13.210780  9883 debug-util.cc:398] Leaking SignalData structure 0x7b08003637e0 after lost signal to thread 9147
I20260709 13:05:13.211018 13201 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:13.165616 12991 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:13.166589 13151 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:13.212383 13201 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.214437 13151 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:13.168115 13152 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.047431  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:13.059155 13279 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.215644 13152 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.063019  9235 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:13.216635  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 169304us
W20260709 13:05:13.216989  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 154069us
W20260709 13:05:13.063707  9263 rpcz_store.cc:269] 0709 13:05:09.122182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.127559 (+  5377us) service_pool.cc:224] Handling call
0709 13:05:11.745643 (+2618084us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b5400355054. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:11.745808 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1033512320}
I20260709 13:05:13.218056 13152 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.218825  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:13.219182 13152 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:13.219471  9263 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:13.219590 13152 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:13.220053 13152 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.220510  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:13.202327 13111 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.065302  9264 raft_consensus.cc:2368] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14.
W20260709 13:05:13.222096  9264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 824) took 4098 ms (client timeout 2002 ms). Trace:
W20260709 13:05:13.222441  9264 rpcz_store.cc:269] 0709 13:05:09.123412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.167900 (+ 44488us) service_pool.cc:224] Handling call
0709 13:05:11.691653 (+2523753us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:05:13.222029 (+1530376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3582336}
I20260709 13:05:13.173360 13002 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.224465  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.227120  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.227246  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.189466 13230 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.230088 13230 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.069038  9115 rpcz_store.cc:276] 0709 13:05:10.754324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.028818 (+2274494us) service_pool.cc:224] Handling call
0709 13:05:13.067654 (+ 38836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.067915 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.067954 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.068011 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:13.068063 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.068138 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.068269 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:13.070734  9266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.121s	user 0.006s	sys 0.009s
I20260709 13:05:13.189668  9856 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:13.071210  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:13.207530  9122 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
W20260709 13:05:13.232136  9265 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1964) took 1662 ms (client timeout 2075 ms). Trace:
W20260709 13:05:13.232234  9122 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2318) took 2283 ms (client timeout 2170 ms). Trace:
I20260709 13:05:13.070731 13278 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.085675  9010 maintenance_manager.cc:419] P cdc5c504a10a4da1812e571d14ae3abd: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10402 bytes on disk
W20260709 13:05:13.083580  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.090932  9272 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.009s	sys 0.007s
I20260709 13:05:13.192220 13233 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:13.198362 13263 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.234483  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1958) took 2383 ms. Trace:
W20260709 13:05:13.088392  9119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.206s	user 0.010s	sys 0.014s
I20260709 13:05:13.234897  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.085235  9268 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:13.199589 13265 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:13.236055  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2519) took 1166 ms. Trace:
I20260709 13:05:13.236162 13263 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:13.083683  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:13.198807  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08001d8620 after lost signal to thread 9020
I20260709 13:05:13.236864  9271 raft_consensus.cc:2376] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9. Re-sending same reply.
I20260709 13:05:13.235575  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 693) took 2635 ms. Trace:
W20260709 13:05:13.237635  9219 debug-util.cc:398] Leaking SignalData structure 0x7b0800168840 after lost signal to thread 9055
I20260709 13:05:13.238421  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2513) took 1170 ms. Trace:
I20260709 13:05:13.237548  9119 rpcz_store.cc:276] 0709 13:05:10.600523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.600642 (+   119us) service_pool.cc:224] Handling call
0709 13:05:10.600907 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.601095 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:10.881975 (+280880us) log.cc:844] Serialized 14 byte log entry
0709 13:05:13.235273 (+2353298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.235323 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:13.235377 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.235435 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.235527 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117242,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":51,"thread_start_us":2629082,"threads_started":2,"wal-append.queue_time_us":434223,"wal-append.run_cpu_time_us":3284,"wal-append.run_wall_time_us":1460762}
I20260709 13:05:13.091411  9860 rpcz_store.cc:276] 0709 13:05:10.316661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.317503 (+   842us) service_pool.cc:224] Handling call
0709 13:05:10.320988 (+  3485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:10.860018 (+539030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.773126 (+1913108us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b540009ae54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:12.773362 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238043520}
I20260709 13:05:13.190371 13231 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.100910 12953 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:13.239202  9009 maintenance_manager.cc:643] P cdc5c504a10a4da1812e571d14ae3abd: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":289,"lbm_reads_lt_1ms":4}
W20260709 13:05:13.092710  9859 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 1120) took 2480 ms (client timeout 1979 ms). Trace:
I20260709 13:05:13.100184 13284 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:13.110862 13156 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:11.439671 13153 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:05:13.091867  9116 rpcz_store.cc:269] 0709 13:05:09.795524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:09.795642 (+   118us) service_pool.cc:224] Handling call
0709 13:05:12.866118 (+3070476us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400285854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:12.866221 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":461822208}
W20260709 13:05:13.091357  9018 rpcz_store.cc:269] 0709 13:04:46.909324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:46.909556 (+   232us) service_pool.cc:224] Handling call
0709 13:05:12.879527 (+25969971us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:01.149748 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:01.168401 (+ 18653us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:01.168433 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:05:01.168451 (+    18us) tablet.cc:662] Decoding operations
0709 13:05:01.184557 (+ 16106us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:01.184628 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:01.184657 (+    29us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:05:01.185858 (+  1201us) tablet.cc:701] Row locks acquired
0709 13:05:01.185878 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:05:01.186024 (+   146us) write_op.cc:270] Start()
0709 13:05:01.186146 (+   122us) write_op.cc:276] Timestamp: P: 1783602301185972 usec, L: 0
0709 13:05:01.186165 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:05:01.186844 (+   679us) log.cc:844] Serialized 7005 byte log entry
0709 13:05:01.696370 (+509526us) op_driver.cc:464] REPLICATION: finished
0709 13:05:02.027534 (+331164us) write_op.cc:301] APPLY: starting
0709 13:05:02.035655 (+  8121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:02.092693 (+ 57038us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:02.104037 (+ 11344us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:02.106307 (+  2270us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:02.106687 (+   380us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:05:02.106745 (+    58us) write_op.cc:312] APPLY: finished
0709 13:05:02.110302 (+  3557us) log.cc:844] Serialized 125 byte log entry
0709 13:05:02.110528 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:02.110733 (+   205us) write_op.cc:454] Released schema lock
0709 13:05:02.110795 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:02.111225 (+   430us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:02.111260 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4154568,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":307972,"apply.run_cpu_time_us":5291,"apply.run_wall_time_us":100070,"num_ops":18,"prepare.queue_time_us":93458,"prepare.run_cpu_time_us":25444,"prepare.run_wall_time_us":125723,"raft.queue_time_us":275803,"raft.run_cpu_time_us":27532,"raft.run_wall_time_us":159830,"replication_time_us":510070,"thread_start_us":550164,"threads_started":4,"wal-append.queue_time_us":125588,"wal-append.run_cpu_time_us":17638,"wal-append.run_wall_time_us":1801021}]]}
I20260709 13:05:13.099956  9020 catalog_manager.cc:5697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 5, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:13.234516 13233 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:13.232532  9265 rpcz_store.cc:269] 0709 13:05:11.569799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.070963 (+1501164us) service_pool.cc:224] Handling call
0709 13:05:13.232076 (+161113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.254213 13233 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:13.239493 12953 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:13.255023  9020 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 4 to 8, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 8 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:13.255646 12953 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:13.256870 12953 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.191242 13232 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.258095  9267 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:05:13.200639  9118 rpcz_store.cc:276] 0709 13:05:10.753919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.754738 (+   819us) service_pool.cc:224] Handling call
0709 13:05:12.987320 (+2232582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:12.987569 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:12.987605 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:12.987661 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:12.987699 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:12.987797 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:12.990003 (+  2206us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b5400285850. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:12.990180 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5133781248}
W20260709 13:05:13.259785  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1940) took 2772 ms (client timeout 1567 ms). Trace:
I20260709 13:05:13.260210 13232 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.260241 13122 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.239411  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.199311  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.261243  9119 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:13.260970 13122 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:13.261474 13156 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:13.200923 13166 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.236474  9268 rpcz_store.cc:276] 0709 13:05:12.069992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.076609 (+1006617us) service_pool.cc:224] Handling call
0709 13:05:13.235999 (+159390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.262845  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2313) took 2470 ms. Trace:
I20260709 13:05:13.263686 13166 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.180810 13182 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:13.133723  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 89709.3us
W20260709 13:05:13.260176  9267 rpcz_store.cc:269] 0709 13:05:10.487622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.584056 (+ 96434us) service_pool.cc:224] Handling call
0709 13:05:13.259563 (+2675507us) spinlock_profiling.cc:243] Waited 2.67 s on lock 0x7b54002a3d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:13.259693 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6147213056}
I20260709 13:05:13.266552  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.266954  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2516) took 1197 ms. Trace:
I20260709 13:05:13.267386 13182 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.194976 13235 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:13.233098 13278 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.300961 13278 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.236340 13265 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.302963 13265 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:13.303476 13200 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.304069 13200 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:13.304399  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 178040us
I20260709 13:05:13.306135 13233 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:13.238533  9219 debug-util.cc:398] Leaking SignalData structure 0x7b080037e080 after lost signal to thread 9114
I20260709 13:05:13.307173  9162 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:13.307472  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08001ce9e0 after lost signal to thread 9147
W20260709 13:05:13.307866  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08003b7aa0 after lost signal to thread 9148
W20260709 13:05:13.309262  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08002cabe0 after lost signal to thread 9222
W20260709 13:05:13.310348  9219 debug-util.cc:398] Leaking SignalData structure 0x7b080040f460 after lost signal to thread 9265
W20260709 13:05:13.310714  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08002c76a0 after lost signal to thread 9270
W20260709 13:05:13.311412  9219 debug-util.cc:398] Leaking SignalData structure 0x7b0800380760 after lost signal to thread 9296
I20260709 13:05:13.114988 12979 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.096025 13269 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.312585  9269 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
W20260709 13:05:13.313905  9269 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1963) took 2338 ms (client timeout 2139 ms). Trace:
W20260709 13:05:13.314219  9269 rpcz_store.cc:269] 0709 13:05:10.975138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.735024 (+759886us) service_pool.cc:224] Handling call
0709 13:05:13.313733 (+1578709us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b540032e454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:13.313843 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545894784}
I20260709 13:05:13.257143  9020 catalog_manager.cc:5697] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 6, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 6 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:13.315287  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08002f6b60 after lost signal to thread 9826
I20260709 13:05:13.315979  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:13.316890  9219 debug-util.cc:398] Leaking SignalData structure 0x7b0800216780 after lost signal to thread 9887
W20260709 13:05:13.264881  9859 rpcz_store.cc:269] 0709 13:05:10.611945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.786033 (+174088us) service_pool.cc:224] Handling call
0709 13:05:12.860611 (+2074578us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b5800108f98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:05:13.092639 (+232028us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5341056}
W20260709 13:05:13.317880  9219 debug-util.cc:398] Leaking SignalData structure 0x7b080038f640 after lost signal to thread 12825
W20260709 13:05:13.318228  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08003bbc80 after lost signal to thread 12869
W20260709 13:05:13.318595  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08002a2ca0 after lost signal to thread 12979
W20260709 13:05:13.318933  9219 debug-util.cc:398] Leaking SignalData structure 0x7b080009d160 after lost signal to thread 12991
I20260709 13:05:13.318389  9020 catalog_manager.cc:5697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:13.319322  9219 debug-util.cc:398] Leaking SignalData structure 0x7b080038ac20 after lost signal to thread 13152
W20260709 13:05:13.319576  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08001f67c0 after lost signal to thread 13153
I20260709 13:05:13.267918 13274 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:13.319851  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08002aad60 after lost signal to thread 13171
I20260709 13:05:13.319937 13274 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:13.320184  9219 debug-util.cc:398] Leaking SignalData structure 0x7b0800388b80 after lost signal to thread 13178
I20260709 13:05:13.320395 13274 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:13.320624  9219 debug-util.cc:398] Leaking SignalData structure 0x7b0800381900 after lost signal to thread 13229
W20260709 13:05:13.320876  9219 debug-util.cc:398] Leaking SignalData structure 0x7b0800343aa0 after lost signal to thread 13230
W20260709 13:05:13.321101  9219 debug-util.cc:398] Leaking SignalData structure 0x7b080040ce80 after lost signal to thread 13232
W20260709 13:05:13.321291  9219 debug-util.cc:398] Leaking SignalData structure 0x7b080015fce0 after lost signal to thread 13233
W20260709 13:05:13.321496  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08003f9420 after lost signal to thread 13235
W20260709 13:05:13.321718  9219 debug-util.cc:398] Leaking SignalData structure 0x7b0800324540 after lost signal to thread 13242
W20260709 13:05:13.322913  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.321918  9219 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5100 after lost signal to thread 13248
I20260709 13:05:13.234858  9272 rpcz_store.cc:276] 0709 13:05:10.851082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.941103 (+ 90021us) service_pool.cc:224] Handling call
0709 13:05:10.941388 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:10.941580 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:10.974826 (+ 33246us) log.cc:844] Serialized 14 byte log entry
0709 13:05:12.102342 (+1127516us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b180004c840. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:05:13.234154 (+1131812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.234203 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:13.234253 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.234321 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.234432 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33067,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":11322,"raft.queue_time_us":7295,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":24908032,"thread_start_us":1131604,"threads_started":1,"wal-append.queue_time_us":1627993}
I20260709 13:05:13.327348  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:13.328258  9272 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:13.328501 13240 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.329018  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:13.329605  9272 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:13.329160 13240 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.330947  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2518) took 1261 ms. Trace:
I20260709 13:05:13.232895  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1160) took 2799 ms. Trace:
I20260709 13:05:13.331897  9856 rpcz_store.cc:276] 0709 13:05:10.433648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.436207 (+  2559us) service_pool.cc:224] Handling call
0709 13:05:10.461717 (+ 25510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.231205 (+2769488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.232688 (+  1483us) spinlock_profiling.cc:243] Waited 2.73 s on lock 0x7b5400716454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:13.232841 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6273373568}
I20260709 13:05:13.333029 13240 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.178705 13273 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.333362  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:13.333544  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.334014  9856 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:13.334180  9859 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:13.218396 13286 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:13.268815  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:13.232777  9122 rpcz_store.cc:269] 0709 13:05:10.948368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.009788 (+1061420us) service_pool.cc:224] Handling call
0709 13:05:13.232155 (+1222367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.263181  9119 rpcz_store.cc:276] 0709 13:05:10.792638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.262086 (+2469448us) service_pool.cc:224] Handling call
0709 13:05:13.262388 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.262555 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.262579 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.262619 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:13.262656 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.262703 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.262795 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:13.335245  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 66496us
I20260709 13:05:13.336158  9119 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:13.336684  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2316) took 2530 ms. Trace:
I20260709 13:05:13.336992  9119 rpcz_store.cc:276] 0709 13:05:10.806738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.335525 (+2528787us) service_pool.cc:224] Handling call
0709 13:05:13.335828 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.336469 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.336635 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.337495 13235 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:13.337996 13235 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.338435 13235 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:13.338965  9122 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:13.339573  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2318) took 2527 ms. Trace:
I20260709 13:05:13.339929 13265 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:13.340431  9122 rpcz_store.cc:276] 0709 13:05:10.812193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.338571 (+2526378us) service_pool.cc:224] Handling call
0709 13:05:13.338816 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.339365 (+   549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.339527 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.342175  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2319) took 2505 ms. Trace:
I20260709 13:05:13.223649 13111 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:13.237200 13270 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:13.120622  9117 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:13.342888 13111 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:05:13.343176  9117 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2321) took 2394 ms (client timeout 1572 ms). Trace:
W20260709 13:05:13.343542  9117 rpcz_store.cc:269] 0709 13:05:10.949077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:11.385075 (+435998us) service_pool.cc:224] Handling call
0709 13:05:13.343107 (+1958032us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.343716 13111 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:13.338232  9119 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:05:13.344568  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.345096  9119 pending_rounds.cc:85] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 8
I20260709 13:05:13.345388  9119 pending_rounds.cc:107] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:05:13.342901 13270 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.346284  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.345847  9020 catalog_manager.cc:5697] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 5, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.347105  9861 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:13.268460  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.269843 13122 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:13.270399 13263 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:13.348472 13270 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.345628  9119 raft_consensus.cc:2889] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:13.349535  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.349937  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.350276  9861 raft_consensus.cc:2393] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:13.253654 13153 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:13.350498  9190 raft_consensus.cc:2376] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11. Re-sending same reply.
I20260709 13:05:13.342487  9122 rpcz_store.cc:276] 0709 13:05:10.836323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.341484 (+2505161us) service_pool.cc:224] Handling call
0709 13:05:13.341716 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.341865 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.341888 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.341924 (+    36us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:13.341982 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.342032 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.342121 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.216629 13279 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.331310  9272 rpcz_store.cc:276] 0709 13:05:12.069760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.330184 (+1260424us) service_pool.cc:224] Handling call
0709 13:05:13.330448 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.330646 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.330671 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.330708 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:13.330738 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.330793 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.330899 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.351370 13153 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.352260  9122 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:13.352761 13153 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.352882  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2316) took 2405 ms. Trace:
I20260709 13:05:13.353092  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2521) took 1282 ms. Trace:
I20260709 13:05:13.260147  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.273200  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.353312  9122 rpcz_store.cc:276] 0709 13:05:10.947871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.351837 (+2403966us) service_pool.cc:224] Handling call
0709 13:05:13.352095 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.352667 (+   572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.352833 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.238816  9271 rpcz_store.cc:276] 0709 13:05:12.068499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.237619 (+1169120us) service_pool.cc:224] Handling call
0709 13:05:13.237921 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.238146 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.238169 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.238204 (+    35us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:13.238231 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.238277 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.238373 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.273527 13231 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.269950  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2314) took 2473 ms. Trace:
I20260709 13:05:13.131398 13282 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.275485  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.357080 13182 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:13.361002 13286 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:13.366463 13182 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:13.367010 13182 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.275810  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.368383 13231 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:13.368701  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.369014 13231 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:13.369401 13231 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:13.369859  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.370359 13231 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.371409  9191 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:13.371522 13231 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:13.372658  9191 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:13.373178  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2511) took 1311 ms. Trace:
I20260709 13:05:13.354560  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.374509  9196 raft_consensus.cc:2376] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9. Re-sending same reply.
W20260709 13:05:13.280609  9089 connection.cc:256] RPC call timeout handler was delayed by -1.2741570099999535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
I20260709 13:05:13.282805  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.366122  9197 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:13.275800  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.284364  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:13.375831  9089 connection.cc:256] RPC call timeout handler was delayed by -1.2507570100000294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
I20260709 13:05:13.288801  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2315) took 2483 ms. Trace:
I20260709 13:05:13.376477  9115 rpcz_store.cc:276] 0709 13:05:10.805455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.288065 (+2482610us) service_pool.cc:224] Handling call
0709 13:05:13.288349 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.288515 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.288538 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.288577 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:13.288606 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.288653 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.288750 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:13.377275  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0023570099999688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
W20260709 13:05:13.377671  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0023570099999688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260709 13:05:13.377731  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.377885  9856 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:13.378563  9115 raft_consensus.cc:2410] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:13.378652  9089 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:13.379371  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:13.211097  9883 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6c20 after lost signal to thread 9148
I20260709 13:05:13.380894 13278 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:13.373507  9116 rpcz_store.cc:276] 0709 13:05:10.797026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.269218 (+2472192us) service_pool.cc:224] Handling call
0709 13:05:13.269491 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.269650 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.269674 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.269712 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:13.269750 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.269799 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.269898 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.295178  9120 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:13.381852  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2522) took 1311 ms. Trace:
I20260709 13:05:13.382766  9271 rpcz_store.cc:276] 0709 13:05:12.070751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.381083 (+1310332us) service_pool.cc:224] Handling call
0709 13:05:13.381367 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.381549 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.381571 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.381609 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:13.381639 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.381694 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.381798 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.384310  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2312) took 2592 ms. Trace:
I20260709 13:05:13.384477  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.384236  9271 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:13.384718  9120 rpcz_store.cc:276] 0709 13:05:10.791976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.037535 (+2245559us) service_pool.cc:224] Handling call
0709 13:05:13.074373 (+ 36838us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b5800000c98. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:05:13.105350 (+ 30977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.382610 (+277260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.384079 (+  1469us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400286254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:13.384239 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509592320}
I20260709 13:05:13.201228  9207 tablet_copy_service.cc:434] P 94504d29c6744f38bb06943479df9466: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-1c199cae5ff14a478726c8a8b491f8f7 on tablet 1c199cae5ff14a478726c8a8b491f8f7 with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:13.354560  9122 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:13.386068  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:13.083655  9123 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2322) took 2134 ms (client timeout 2312 ms). Trace:
W20260709 13:05:13.386641  9123 rpcz_store.cc:269] 0709 13:05:10.949325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:12.837220 (+1887895us) service_pool.cc:224] Handling call
0709 13:05:13.083598 (+246378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.386967  9120 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 12 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:05:13.387027  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 50382us
I20260709 13:05:13.387637  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.306571 13200 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.388286  9123 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
W20260709 13:05:13.382247  9883 debug-util.cc:398] Leaking SignalData structure 0x7b0800382120 after lost signal to thread 9222
W20260709 13:05:13.389770  9883 debug-util.cc:398] Leaking SignalData structure 0x7b08003efa20 after lost signal to thread 9296
I20260709 13:05:13.390723  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.316926  9269 raft_consensus.cc:2410] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:13.267323  9267 rpcz_store.cc:276] 0709 13:05:12.069243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.266150 (+1196907us) service_pool.cc:224] Handling call
0709 13:05:13.266439 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.266639 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.266673 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.266712 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:13.266747 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.266799 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.266894 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.392354  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2524) took 1319 ms. Trace:
I20260709 13:05:13.261336  9114 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:13.188391 13229 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:13.366091  9118 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:13.392730  9267 rpcz_store.cc:276] 0709 13:05:12.073375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.391597 (+1318222us) service_pool.cc:224] Handling call
0709 13:05:13.391888 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.392050 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.392072 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.392110 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:13.392140 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.392190 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.392297 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.393410  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.394196  9195 raft_consensus.cc:2410] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:13.374223 13282 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.394409  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2526) took 1320 ms. Trace:
I20260709 13:05:13.349653  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2317) took 2539 ms. Trace:
I20260709 13:05:13.394795  9267 rpcz_store.cc:276] 0709 13:05:12.074051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.393708 (+1319657us) service_pool.cc:224] Handling call
0709 13:05:13.393975 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.394120 (+   145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.394145 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.394182 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:13.394212 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.394265 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.394363 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.392486  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2525) took 1318 ms. Trace:
I20260709 13:05:13.397761  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.398007  9267 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:13.397930  9269 rpcz_store.cc:276] 0709 13:05:12.073788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.391667 (+1317879us) service_pool.cc:224] Handling call
0709 13:05:13.391982 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.392158 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.392183 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.392222 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:13.392259 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.392320 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.392434 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.398617  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2527) took 1324 ms. Trace:
I20260709 13:05:13.093469  9121 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:13.399287  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.399507 13234 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.399642  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2528) took 1325 ms. Trace:
I20260709 13:05:13.399997 13234 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.400120  9269 rpcz_store.cc:276] 0709 13:05:12.074571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.398965 (+1324394us) service_pool.cc:224] Handling call
0709 13:05:13.399211 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.399350 (+   139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.399372 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.399415 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:13.399444 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.399492 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.399590 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:13.401232  9121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 2310) took 2626 ms (client timeout 1753 ms). Trace:
I20260709 13:05:13.401819  9118 raft_consensus.cc:2410] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:05:13.333871 13273 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.382283 13274 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.404172  9859 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:13.399015  9267 rpcz_store.cc:276] 0709 13:05:12.074325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.397631 (+1323306us) service_pool.cc:224] Handling call
0709 13:05:13.397860 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.398414 (+   554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.398562 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.379782  9020 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 4 to 5, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.314440 13269 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.405570 13273 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.405903  9020 catalog_manager.cc:5697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:13.407150 13274 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:13.385282  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2523) took 1312 ms. Trace:
I20260709 13:05:13.386255  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2317) took 2438 ms. Trace:
I20260709 13:05:13.268100  9194 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:05:13.390964  9883 debug-util.cc:398] Leaking SignalData structure 0x7b0800383ae0 after lost signal to thread 9887
I20260709 13:05:13.394956  9119 rpcz_store.cc:276] 0709 13:05:10.810312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.337664 (+2527352us) service_pool.cc:224] Handling call
0709 13:05:13.337931 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.349426 (+ 11495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.349600 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:13.408145  9883 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8c60 after lost signal to thread 12825
I20260709 13:05:13.387312 13298 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:13.396093 13282 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.396263  9120 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:13.408475  9271 rpcz_store.cc:276] 0709 13:05:12.073104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.383724 (+1310620us) service_pool.cc:224] Handling call
0709 13:05:13.384054 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.385077 (+  1023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.385233 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.408929  9119 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:05:13.409406  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2319) took 2460 ms. Trace:
I20260709 13:05:13.396659  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.410122  9115 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:13.365394 13152 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:13.409787  9120 rpcz_store.cc:276] 0709 13:05:10.948605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.387613 (+2439008us) service_pool.cc:224] Handling call
0709 13:05:13.395893 (+  8280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.409182 (+ 13289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.409357 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.411104  9265 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:13.289760  9826 connection.cc:256] RPC call timeout handler was delayed by -2.4403545509999276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
I20260709 13:05:13.411955  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2320) took 2463 ms. Trace:
W20260709 13:05:13.412062  9826 leader_election.cc:336] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.187s (SENT)
I20260709 13:05:13.353353 13279 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.367349 13286 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:13.413614  9119 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:13.412385  9123 rpcz_store.cc:276] 0709 13:05:10.948841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.389201 (+2440360us) service_pool.cc:224] Handling call
0709 13:05:13.389539 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:13.389907 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.389933 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:13.390504 (+   571us) log.cc:844] Serialized 52 byte log entry
0709 13:05:13.411634 (+ 21130us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:13.411674 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.411734 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.411794 (+    60us) raft_consensus.cc:1685] finished
0709 13:05:13.411804 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.411906 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11857,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":20754,"threads_started":1,"wal-append.queue_time_us":1922}
I20260709 13:05:13.415047  9119 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:05:13.415401  9119 pending_rounds.cc:85] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 9
I20260709 13:05:13.401449  9116 raft_consensus.cc:2410] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:05:13.401687  9121 rpcz_store.cc:269] 0709 13:05:10.775094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.941237 (+166143us) service_pool.cc:224] Handling call
0709 13:05:13.401164 (+2459927us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.379890 13095 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.380330 13240 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:13.263556  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2515) took 1194 ms. Trace:
I20260709 13:05:13.412533  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:13.364184  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 57075.2us
I20260709 13:05:13.351555  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.415665  9119 pending_rounds.cc:107] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:05:13.458000 13152 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:13.459399 13233 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:13.459816 13298 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:13.460186  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:13.409808  9114 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:13.461179 13279 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:13.460690 13298 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.461549  9117 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:13.461688 13279 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:13.462133 13298 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.462657  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2320) took 2620 ms. Trace:
I20260709 13:05:13.420722 13234 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.462604 13279 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.463290  9117 rpcz_store.cc:276] 0709 13:05:10.842393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.344288 (+2501895us) service_pool.cc:224] Handling call
0709 13:05:13.461388 (+117100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.461899 (+   511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.462485 (+   586us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540034e750. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:13.462604 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268662656}
I20260709 13:05:13.464438  9123 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:13.442433  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.401506  9269 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:13.465562  9195 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.431138  9115 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:13.412415  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2514) took 1343 ms. Trace:
I20260709 13:05:13.463565  9268 rpcz_store.cc:276] 0709 13:05:12.068999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.262822 (+1193823us) service_pool.cc:224] Handling call
0709 13:05:13.263102 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.263264 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.263291 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.263331 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:13.263362 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.263411 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.263501 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.468381  9265 rpcz_store.cc:276] 0709 13:05:12.068763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.254429 (+1185666us) service_pool.cc:224] Handling call
0709 13:05:13.254758 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.411549 (+156791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.412198 (+   649us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54002a3854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:13.412363 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359075072}
I20260709 13:05:13.469895 13240 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.470456 13240 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:13.408871  9122 rpcz_store.cc:276] 0709 13:05:10.948111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.354167 (+2406056us) service_pool.cc:224] Handling call
0709 13:05:13.354414 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.386018 (+ 31604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.386197 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:13.449896  9089 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.521s (SENT)
I20260709 13:05:13.458281  9119 raft_consensus.cc:2889] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:13.473922  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.440519  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.475411 13095 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.477095  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:13.477555  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.477954 13095 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.375440  9272 rpcz_store.cc:276] 0709 13:05:12.070504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.352374 (+1281870us) service_pool.cc:224] Handling call
0709 13:05:13.352640 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.352804 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.352832 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:13.352873 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:13.352902 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.352953 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.353045 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.473212  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2512) took 1410 ms. Trace:
I20260709 13:05:13.482463  9264 rpcz_store.cc:276] 0709 13:05:12.062678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.223204 (+1160526us) service_pool.cc:224] Handling call
0709 13:05:13.223507 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.223861 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.223889 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.224351 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.472903 (+248552us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:13.472942 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.473006 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.473048 (+    42us) raft_consensus.cc:1685] finished
0709 13:05:13.473058 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.473161 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4384,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"thread_start_us":248211,"threads_started":1,"wal-append.queue_time_us":137819}
W20260709 13:05:13.488534  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.466387  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2529) took 1391 ms. Trace:
I20260709 13:05:13.489953  9269 rpcz_store.cc:276] 0709 13:05:12.074818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.401081 (+1326263us) service_pool.cc:224] Handling call
0709 13:05:13.401332 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:13.466167 (+ 64835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.466338 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.489514 13234 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:13.492369 13182 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.496152  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2517) took 1426 ms. Trace:
I20260709 13:05:13.496592  9270 rpcz_store.cc:276] 0709 13:05:12.069531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.077466 (+1007935us) service_pool.cc:224] Handling call
0709 13:05:13.496085 (+418619us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:13.498598 13235 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.500917 13235 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:13.373612  9263 rpcz_store.cc:276] 0709 13:05:12.062340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.221419 (+1159079us) service_pool.cc:224] Handling call
0709 13:05:13.228846 (+  7427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.229199 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.229228 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.229654 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.371951 (+142297us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:13.371987 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.372048 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.372089 (+    41us) raft_consensus.cc:1685] finished
0709 13:05:13.372101 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.372946 (+   845us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b54000fafd0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:13.373126 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1272,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"spinlock_wait_cycles":16343296,"thread_start_us":141938,"threads_started":1,"wal-append.queue_time_us":137134}
I20260709 13:05:13.502878 13111 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.503403 13111 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.505465 13111 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:13.506134 13231 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:13.507009  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 94975.8us
W20260709 13:05:13.507470  9826 connection.cc:256] RPC call timeout handler was delayed by -2.2801545510000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
I20260709 13:05:13.498842  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:13.508052  9826 connection.cc:256] RPC call timeout handler was delayed by -2.104730027999949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 13:05:13.508426  9826 connection.cc:256] RPC call timeout handler was delayed by -2.0260450879999325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
W20260709 13:05:13.509066  9826 connection.cc:256] RPC call timeout handler was delayed by -2.0123330219998934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25070000000000003s for a response
I20260709 13:05:13.509097 13234 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:13.509533  9826 connection.cc:256] RPC call timeout handler was delayed by -1.9765792169999941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:05:13.510066  9826 connection.cc:256] RPC call timeout handler was delayed by -1.4749503829999639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
W20260709 13:05:13.510550  9826 connection.cc:256] RPC call timeout handler was delayed by -1.4149792169999955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2257s for a response
I20260709 13:05:13.510663 13290 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:13.511495 13290 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.512121 13233 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.513337 13233 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:13.510903  9826 connection.cc:256] RPC call timeout handler was delayed by -1.4115995759999578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3574s for a response
W20260709 13:05:13.515029  9826 connection.cc:256] RPC call timeout handler was delayed by -1.2700450879999607s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28290000000000004s for a response
W20260709 13:05:13.515693  9826 connection.cc:256] RPC call timeout handler was delayed by -1.2331067129999838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
I20260709 13:05:13.512450 13298 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:13.520720  9089 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:13.516196  9826 connection.cc:256] RPC call timeout handler was delayed by -1.1822481199999402s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
I20260709 13:05:13.521507  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:13.522015  9826 connection.cc:256] RPC call timeout handler was delayed by -1.1713545509999221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3083s for a response
I20260709 13:05:13.522199  9856 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:13.517907  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:13.518174  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.518672  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:13.524375  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56502.5us
W20260709 13:05:13.520728  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 1.7597e+06us
I20260709 13:05:13.496176  9195 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:13.523851  9190 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
W20260709 13:05:13.534538  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 62319.2us
W20260709 13:05:13.520115  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 1.94957e+06us
I20260709 13:05:13.535557  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.529553  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:13.536746  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.538717  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.540378  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.525959  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.541014  9855 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:13.541661  9825 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:13.541949  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.538666 13182 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:13.542583  9859 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:13.555967  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.556910  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.557687  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.557596  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:13.514670 13290 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:13.559650  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 716) took 3255 ms. Trace:
I20260709 13:05:13.566655  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.561017  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1946) took 2938 ms. Trace:
I20260709 13:05:13.566639  9858 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:13.568279  9856 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:13.566807  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:13.567104  9860 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:13.572870  9161 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:13.567621  9266 rpcz_store.cc:276] 0709 13:05:10.622685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.863407 (+240722us) service_pool.cc:224] Handling call
0709 13:05:10.863687 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:10.866602 (+  2915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:10.866653 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:10.867551 (+   898us) log.cc:844] Serialized 26 byte log entry
0709 13:05:13.231023 (+2363472us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:13.472182 (+241159us) log.cc:844] Serialized 14 byte log entry
0709 13:05:13.560699 (+ 88517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.560774 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.560824 (+    50us) raft_consensus.cc:1685] finished
0709 13:05:13.560835 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.560961 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5249,"prepare.queue_time_us":102419,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":9878,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":2604580,"threads_started":3,"wal-append.queue_time_us":443676,"wal-append.run_cpu_time_us":7489,"wal-append.run_wall_time_us":1457095}
I20260709 13:05:13.544832 13278 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:13.544523 13277 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:13.575860  9087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.044s	sys 0.049s
I20260709 13:05:13.541522 13234 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:13.567255  9189 rpcz_store.cc:276] 0709 13:05:10.304331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:10.316114 (+ 11783us) service_pool.cc:224] Handling call
0709 13:05:10.316479 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:10.345374 (+ 28895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:10.345418 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:10.350660 (+  5242us) log.cc:844] Serialized 52 byte log entry
0709 13:05:10.956156 (+605496us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b180032bf00. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:05:13.054570 (+2098414us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:13.133505 (+ 78935us) log.cc:844] Serialized 14 byte log entry
0709 13:05:13.542639 (+409134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:13.542857 (+   218us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:13.542905 (+    48us) raft_consensus.cc:1685] finished
0709 13:05:13.559350 (+ 16445us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:13.559570 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31298,"prepare.queue_time_us":75603,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":88473,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":17723,"spinlock_wait_cycles":27836800,"thread_start_us":2554120,"threads_started":3,"wal-append.queue_time_us":1318103,"wal-append.run_cpu_time_us":8544,"wal-append.run_wall_time_us":1409408}
I20260709 13:05:13.542824 13257 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:13.580687  9087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.044s	sys 0.050s
W20260709 13:05:13.588990  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 538974us
W20260709 13:05:13.589581  9087 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.505s (SENT)
I20260709 13:05:13.589936  9087 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:13.590741  9087 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.521s (SENT)
I20260709 13:05:13.590978  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:13.582671 13263 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:13.583482  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:13.592092  9087 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.555s (SENT)
I20260709 13:05:13.586712  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:13.562891 13269 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:13.592712  9087 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:13.597796  9087 leader_election.cc:336] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.689s (SENT)
I20260709 13:05:13.614029  9828 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:13.600149 13263 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:13.616928 13152 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.618405 13231 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:13.541889 13095 consensus_queue.cc:1059] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:05:13.596565 13166 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:13.622692  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:13.596410 13276 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:13.612972 13180 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:13.603544  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:13.625275  9087 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.853s (SENT)
I20260709 13:05:13.575014 13284 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:13.626773  9087 leader_election.cc:336] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.865s (SENT)
I20260709 13:05:13.627346  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:13.578605 13168 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:13.563995 13151 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:05:13.564190 13201 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 13:05:13.654102  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 68162.7us
I20260709 13:05:13.680958  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:13.658108  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 71499us
I20260709 13:05:13.565860 13265 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:05:13.587813 13278 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:13.695068  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 133537us
I20260709 13:05:13.579387 13230 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:13.602258 13275 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:13.577307 13286 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.103s
W20260709 13:05:13.679267  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 75847.3us
I20260709 13:05:13.602479 13184 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:13.654311 13233 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:13.612819 13257 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.688927 13166 raft_consensus.cc:3055] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:05:13.689558  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 66934.9us
I20260709 13:05:13.596184 13111 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.099s
W20260709 13:05:13.691077  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 64338.6us
I20260709 13:05:13.693145  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.691711 13171 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:13.614275 13234 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:13.723732  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.725641  9122 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:13.594359 13290 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:13.629953 13284 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.631059  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:13.716486 13278 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:13.690224 13276 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:13.719990  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.694114 13235 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:13.681389 13229 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:13.624583 13182 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
W20260709 13:05:13.721894  9087 leader_election.cc:336] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.915s (SENT)
I20260709 13:05:13.744851  9087 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:13.723670  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.746057  9114 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:13.679316 13168 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.728893  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:13.774041 13168 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:13.774537  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45770.5us
I20260709 13:05:13.774936 13002 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:13.774658 13234 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.775420  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.776177  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.776546  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:13.697314  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 151738us
I20260709 13:05:13.696978 13275 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.777827  9853 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:13.778196 13275 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.778496  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.778807  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:13.729017  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 98039.9us
I20260709 13:05:13.731860  9116 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:13.617623 13263 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:13.730551  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 49714.7us
I20260709 13:05:13.717703 13184 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.730448 13284 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.739316 13156 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:13.788331  9192 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:13.719822 13166 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:13.753757 13298 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.755143 13151 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:13.732692 13276 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.733109 13269 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:13.756335 13265 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:13.757755  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.802429  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:13.762866 13201 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:13.760375 12953 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
W20260709 13:05:13.766108  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49760us
I20260709 13:05:13.767225  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:13.739632 13235 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.769037  9114 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:13.769285  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:13.770238  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48384.4us
I20260709 13:05:13.822324  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.823030  9115 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:13.823275  9087 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.065s (SENT)
I20260709 13:05:13.823726  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:13.825115 13195 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:13.776423  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.827418 13002 consensus_queue.cc:1059] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:13.780031  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.732854  9119 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:13.828261 13002 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:13.802094 13263 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.828969 13235 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:13.830632 13002 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:05:13.832831 13308 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:05:13.833292 13282 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:13.833726 13282 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:13.803503 13298 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.834542 13282 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.781293  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.781378  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:13.782390  9826 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.673s (SENT)
I20260709 13:05:13.836766  9116 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:13.836987  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.837188  9858 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:05:13.837865  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 55509.5us
I20260709 13:05:13.829013  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:13.838559  9826 leader_election.cc:336] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.686s (SENT)
I20260709 13:05:13.838973  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.784544  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:13.840287  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 55787.3us
I20260709 13:05:13.840695 13122 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:13.842765  9115 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:13.789407  9122 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:13.731945 13290 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.844762 13290 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.845503 13282 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:13.846067  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.846854 13269 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:13.791265 13095 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.787714 13288 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.788971 13156 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:13.807848 13265 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:13.810057 13166 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.803860 13276 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.849505 13166 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:13.803236 13151 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.851941 13289 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:13.852880 13265 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.853741  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.854801  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.855971 13288 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:13.856323  9826 leader_election.cc:336] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.927s (SENT)
I20260709 13:05:13.856491 13288 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:13.856966 13288 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.787919 13152 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.795239 13184 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:13.837922 13233 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:13.868517  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44014.8us
W20260709 13:05:13.783723  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.852017 13151 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.856806  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:13.865639  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.898284  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.898876 13201 consensus_queue.cc:1059] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:13.899521  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.899760 13314 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:13.900050 13201 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:13.900174  9114 raft_consensus.cc:2376] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8. Re-sending same reply.
I20260709 13:05:13.900588 13201 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:13.900713 13314 raft_consensus.cc:3055] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:13.901122 13314 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:13.901811 12953 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:13.901473 13201 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.902169 12953 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:13.902966 12953 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.904127 13207 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:13.905063 13207 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:13.906168 13277 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:13.907009 13279 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:13.907403 13279 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:13.908174 13279 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:13.909283  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:13.909473 13279 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:13.909890  9267 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:05:13.910899  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:13.911463  9266 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:13.912753  9263 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:05:13.912896  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56600.8us
W20260709 13:05:13.913625  9826 leader_election.cc:336] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.989s (SENT)
W20260709 13:05:13.912813  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.913996  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.912739  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:13.915047  9826 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.228s (SENT)
I20260709 13:05:13.915915 13277 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:13.916100  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.916260 13314 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.917512 13314 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:13.917719 12953 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:13.818418  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.836393 13298 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:13.919559  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:13.919941  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 138644us
I20260709 13:05:13.926741  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:13.920318  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080003cf80 after lost signal to thread 8024
I20260709 13:05:13.927659  9116 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
W20260709 13:05:13.927699  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08000f0e60 after lost signal to thread 9020
I20260709 13:05:13.855968 13269 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:13.901122  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.916749  9854 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
W20260709 13:05:13.934546  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.934662  9114 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:13.935189  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.935776  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.921058  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:13.936966  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.923002  9856 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:13.924386  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.938697  9853 raft_consensus.cc:2410] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:13.924916  9860 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:13.855361 13168 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:13.931599  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080003caa0 after lost signal to thread 9055
W20260709 13:05:13.946635  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080003ce00 after lost signal to thread 9115
W20260709 13:05:13.920746  9826 leader_election.cc:336] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.257s (SENT)
I20260709 13:05:13.866962 13002 raft_consensus.cc:3055] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:13.815615 13275 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.939595  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:13.955011  9263 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:13.847422 13240 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.928572  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:13.947409  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:13.962549  9116 raft_consensus.cc:2376] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7. Re-sending same reply.
W20260709 13:05:13.963054  9826 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.507s (SENT)
I20260709 13:05:13.963635  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:13.964270 13265 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:13.946962  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08000c28e0 after lost signal to thread 9117
W20260709 13:05:13.965109  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080003ca80 after lost signal to thread 9121
I20260709 13:05:13.964839  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:13.965706  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080025c660 after lost signal to thread 9147
W20260709 13:05:13.966009  9293 debug-util.cc:398] Leaking SignalData structure 0x7b0800240700 after lost signal to thread 9148
W20260709 13:05:13.966337  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08000a4e80 after lost signal to thread 9163
W20260709 13:05:13.967304  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003a47c0 after lost signal to thread 9208
W20260709 13:05:13.967728  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003a49c0 after lost signal to thread 9222
W20260709 13:05:13.968051  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003a4a80 after lost signal to thread 9235
W20260709 13:05:13.968417  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003a4b00 after lost signal to thread 9237
W20260709 13:05:13.968621  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003a4ac0 after lost signal to thread 9238
W20260709 13:05:13.969089  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1940 after lost signal to thread 9263
W20260709 13:05:13.969480  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1a60 after lost signal to thread 9264
W20260709 13:05:13.969751  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1a80 after lost signal to thread 9265
W20260709 13:05:13.970017  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003a26a0 after lost signal to thread 9266
W20260709 13:05:13.970293  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1ae0 after lost signal to thread 9268
W20260709 13:05:13.970643  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1aa0 after lost signal to thread 9269
W20260709 13:05:13.970881  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003b9fe0 after lost signal to thread 9270
I20260709 13:05:13.970976  9857 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 13:05:13.964867  9826 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 3.163s (SENT)
I20260709 13:05:13.972702  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:13.973714 13265 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:13.957456  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.978989  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.958510 13270 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:13.980294 13270 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:13.983212 13270 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:13.958724  9238 connection.cc:256] RPC call timeout handler was delayed by -3.8733507969999437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
I20260709 13:05:13.959106 13153 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:13.928825 12979 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:05:13.971092  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1b00 after lost signal to thread 9271
I20260709 13:05:13.948484  9862 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:14.014174  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08000b1fe0 after lost signal to thread 9296
W20260709 13:05:14.014591  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080038fe00 after lost signal to thread 9825
W20260709 13:05:14.014891  9293 debug-util.cc:398] Leaking SignalData structure 0x7b0800259940 after lost signal to thread 9826
I20260709 13:05:14.014897  9263 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:14.016037  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:14.017911  9293 debug-util.cc:398] Leaking SignalData structure 0x7b0800164fa0 after lost signal to thread 9887
W20260709 13:05:14.018599  9293 debug-util.cc:398] Leaking SignalData structure 0x7b0800247a00 after lost signal to thread 12825
W20260709 13:05:14.018962  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080035fbe0 after lost signal to thread 12833
W20260709 13:05:14.019232  9293 debug-util.cc:398] Leaking SignalData structure 0x7b0800397c20 after lost signal to thread 12869
W20260709 13:05:14.019537  9293 debug-util.cc:398] Leaking SignalData structure 0x7b0800377d80 after lost signal to thread 12979
W20260709 13:05:14.019851  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08002a7500 after lost signal to thread 12991
W20260709 13:05:14.020246  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003704c0 after lost signal to thread 13152
W20260709 13:05:14.020628  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003e0a20 after lost signal to thread 13153
I20260709 13:05:13.993851  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.988718  9860 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:13.985539  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:13.920269 13289 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:05:14.027026  9114 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:14.027257  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003077a0 after lost signal to thread 13171
W20260709 13:05:14.027541  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080037ac80 after lost signal to thread 13178
W20260709 13:05:14.027801  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080029dfc0 after lost signal to thread 13182
W20260709 13:05:13.983603  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:13.940191 13269 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:14.007169  9238 connection.cc:256] RPC call timeout handler was delayed by -3.8598507970000355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
W20260709 13:05:14.031065  9238 connection.cc:256] RPC call timeout handler was delayed by -3.8301507969999875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
I20260709 13:05:14.021490  9853 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:14.023396  9120 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 9. Ops: []
W20260709 13:05:14.031527  9238 connection.cc:256] RPC call timeout handler was delayed by -3.5988507969999546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1896s for a response
W20260709 13:05:14.032995  9238 connection.cc:256] RPC call timeout handler was delayed by -3.5808507970000392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 13:05:14.033385  9238 connection.cc:256] RPC call timeout handler was delayed by -3.5718507970000246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1926s for a response
W20260709 13:05:14.033747  9238 connection.cc:256] RPC call timeout handler was delayed by -3.5709507970000232s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
W20260709 13:05:14.034128  9238 connection.cc:256] RPC call timeout handler was delayed by -3.5304507969999577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
W20260709 13:05:14.023470  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 102591us
I20260709 13:05:14.036254  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:13.946916 13232 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:14.028046  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08001462c0 after lost signal to thread 13195
I20260709 13:05:13.954044 13002 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:05:14.012116 13294 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:05:14.013101 12979 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:14.040145  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 99116.7us
W20260709 13:05:14.042613  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08000ab500 after lost signal to thread 13207
W20260709 13:05:14.045158  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003fad40 after lost signal to thread 13229
W20260709 13:05:14.045408  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003f7420 after lost signal to thread 13230
W20260709 13:05:14.046729  9293 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6260 after lost signal to thread 13232
W20260709 13:05:14.046998  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080028e100 after lost signal to thread 13233
I20260709 13:05:14.046861  9116 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:05:14.047459  9293 debug-util.cc:398] Leaking SignalData structure 0x7b0800375e20 after lost signal to thread 13235
I20260709 13:05:14.035254 13269 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:14.038079 13318 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:05:13.987444 13207 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 13:05:14.055187  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.034978  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:14.056613 13002 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:14.064280 12800 tablet_copy_client.cc:538] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:05:14.017228 13153 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.064831 13168 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:14.066440 13153 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:14.031905 13201 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:14.068216  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:14.069232  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.070272  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.041503 13324 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:14.046357  9162 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:14.047730  9293 debug-util.cc:398] Leaking SignalData structure 0x7b0800202280 after lost signal to thread 13242
I20260709 13:05:14.041240  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:14.037602 13111 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:14.038798  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:14.084335  9857 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:14.074270  9089 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:14.081189 13111 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:14.076886  9293 debug-util.cc:398] Leaking SignalData structure 0x7b080036fc60 after lost signal to thread 13248
I20260709 13:05:14.180919  9862 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:14.183459  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 100172us
W20260709 13:05:14.190583  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 116382us
I20260709 13:05:14.188192  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:14.191502  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.192850  9115 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:14.073252 13324 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.073000 13201 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.048661 13289 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.139s
I20260709 13:05:14.185158 13168 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:14.185299 13273 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:14.181110 13095 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:14.046047 13230 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.142s
W20260709 13:05:14.228464  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.235256  9856 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:14.229310 12833 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:14.230985  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.053282 13184 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:14.239090 13323 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
W20260709 13:05:14.240289 13319 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:05:14.219892 13201 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:14.254381  9862 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:14.243270 13257 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:14.245005  9120 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:14.239158  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:14.214756 13195 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:14.220534 13324 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:14.255573  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:14.260653  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.262216  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.263064  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.263856  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.261559  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:14.271651  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:14.314337  9195 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:05:14.315455  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:14.316121  9855 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:14.318339  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:14.319947  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.321681  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:14.325590  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.330516  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:14.349573 13184 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:14.311937 12833 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:14.356299 13293 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:05:14.356630 12833 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:14.358069 13290 consensus_meta.cc:287] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.512s	user 0.005s	sys 0.003s
I20260709 13:05:14.358496 13290 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.360205  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.363343  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:14.366297  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.360109  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:14.374080  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 187649us
I20260709 13:05:14.380565  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.384809  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:14.387430  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 195985us
I20260709 13:05:14.393471  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.397092  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:14.399312  9120 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: [9.11-11.12]
I20260709 13:05:14.399273  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:14.402460  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.403148  9196 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:14.400075  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:14.255631 13200 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.126s
I20260709 13:05:14.372910 13273 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:14.412416  9198 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:14.407963  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.327041 13195 raft_consensus.cc:697] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:14.406309  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:14.409087  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.401309 13095 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:14.413786  9862 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:05:14.422792  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.423323  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:14.406209 13287 log.cc:927] Time spent T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:05:14.415720 13234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.015s	sys 0.014s
I20260709 13:05:14.420392  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:14.408736 13288 consensus_meta.cc:287] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.551s	user 0.023s	sys 0.008s
I20260709 13:05:14.418262  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:14.430102  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.409438 13275 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:14.432705  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:14.439889  9853 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:14.416591 13166 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.003s
I20260709 13:05:14.408924  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:46626
W20260709 13:05:14.430826 13234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.015s	sys 0.014s
I20260709 13:05:14.444924  9121 raft_consensus.cc:2410] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:14.445227  9193 raft_consensus.cc:606] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:14.436654  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:14.422583 12953 consensus_meta.cc:287] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.003s
I20260709 13:05:14.450815  9853 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:05:14.456506  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.457165  9825 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:14.409585 13269 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:14.467227  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.431423 13240 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:14.483541  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.489861  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:14.471626 13151 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.559s	user 0.006s	sys 0.000s
W20260709 13:05:14.501251  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 44154.9us
I20260709 13:05:14.502004  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:05:14.472914 13235 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.643s	user 0.008s	sys 0.008s
I20260709 13:05:14.502606 13151 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.473335 13274 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:14.520030  9191 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:14.442585 13288 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:14.530719 13151 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:14.527012 13182 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:14.532971 13182 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:14.534202 13182 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.539924  9196 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:14.544466 13095 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:14.545545 12953 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:14.548409 13342 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:14.549572 12800 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:05:14.495167 13327 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:05:14.581672 13002 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:14.589798 13284 consensus_meta.cc:287] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.773s	user 0.004s	sys 0.005s
I20260709 13:05:14.590299 13284 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:14.553234 13325 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:05:14.592497  9119 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:14.578534 13314 consensus_meta.cc:287] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.000s
I20260709 13:05:14.557814 13232 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:14.595177 13232 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:14.596191 13232 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.510946 13235 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:14.597366 13307 log.cc:927] Time spent T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.905s	user 0.009s	sys 0.000s
I20260709 13:05:14.598104  9855 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:14.510804 13290 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:14.511041 13195 consensus_queue.cc:237] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:14.474727 13201 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:14.570204 13279 consensus_meta.cc:287] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.660s	user 0.008s	sys 0.002s
I20260709 13:05:14.572672 13286 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:14.579810 13269 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:14.519937 13277 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:14.547408 12953 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:05:14.575503 13156 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:14.616185 13111 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.616899 12953 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:14.617071 13156 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:14.617349 12953 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:14.617600 13111 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:14.618291 12953 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.594596  9122 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:05:14.599493 13235 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:14.557339 13276 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.708s	user 0.006s	sys 0.009s
W20260709 13:05:14.604809  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 138304us
I20260709 13:05:14.606312 12833 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:14.613327 13201 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:14.604719 13286 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:14.614777  9192 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:14.661013  9120 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:14.661180  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 626295us
W20260709 13:05:14.603436  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 51894.2us
I20260709 13:05:14.624485  9115 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:05:14.662367  9164 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:14.629825 13337 consensus_queue.cc:1059] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:14.629916 13273 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:14.634333 13282 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:14.634658  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 446498us
I20260709 13:05:14.664515  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:14.665548  9235 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:14.634949 13153 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:14.641206 13299 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:14.620937 13184 raft_consensus.cc:1064] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:05:14.648553  9122 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:14.649572  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:14.693398  9119 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:14.619683 12953 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:14.668730 13279 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:05:14.656968  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:14.657380  9123 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:14.629034 13338 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:14.662546  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:14.625463 13278 consensus_meta.cc:287] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.840s	user 0.009s	sys 0.001s
W20260709 13:05:14.669621  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.061s	sys 0.095s
W20260709 13:05:14.714469  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.061s	sys 0.095s
W20260709 13:05:14.714929  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 675975us
W20260709 13:05:14.717971  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 250822us
W20260709 13:05:14.679028  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.684823  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:14.686508  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.686935 13269 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:05:14.734627  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 66727.5us
I20260709 13:05:14.690377 13153 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:14.684675 13282 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:14.691619 13342 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:14.697329  9194 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
W20260709 13:05:14.750440  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.691289 13152 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.743448  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.691079 13002 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:14.685071 13299 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:14.761797  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:14.777184  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 115198us
I20260709 13:05:14.782711  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.666612 13257 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:05:14.754380  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:14.785132  9264 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.661828 13276 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.781296  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:14.791378  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.795702  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.802755  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:14.803017  9161 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:14.805387 13343 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:14.736020 13336 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:14.806881  9267 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:14.753355 13122 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:14.824877  9117 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:14.816975 13349 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:14.826665 13232 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.827083 13349 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:14.788887 13270 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.806057 13168 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:14.761014 13171 raft_consensus.cc:3055] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:14.830127  9123 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:05:14.830603  9123 pending_rounds.cc:85] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 7
I20260709 13:05:14.830889  9123 pending_rounds.cc:107] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:05:14.831162  9123 raft_consensus.cc:2889] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:14.810667  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:14.812503  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:14.807969  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.832342  9862 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:14.753132 13282 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:14.820052  9853 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:14.821287  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:14.836601  9269 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:14.822165  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:14.837559  9858 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:14.839184  9856 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:14.763976 13337 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:14.833474  9825 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:14.841920  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:14.733029 13278 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:14.849407  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 113088us
I20260709 13:05:14.842314 13201 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:14.835189 13233 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.941s	user 0.011s	sys 0.000s
W20260709 13:05:14.850900  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 445909us
I20260709 13:05:14.787986 13152 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.814778 13265 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:05:14.856722  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:14.840291 13282 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:14.840019 13347 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:14.855827  9825 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.869709  9825 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:14.859910  9120 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:14.836434 13171 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:14.836125 13284 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:14.864120  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 61178.9us
I20260709 13:05:14.864923  9118 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:05:14.756696 12979 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:14.828030 13349 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.841730 13346 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:05:14.880053 13207 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:14.887490  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 711) took 1109 ms. Trace:
I20260709 13:05:14.900771  9117 rpcz_store.cc:276] 0709 13:05:13.777797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.778595 (+   798us) service_pool.cc:224] Handling call
0709 13:05:14.887323 (+1108728us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400284e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:14.887440 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2403902848}
I20260709 13:05:14.890843  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:14.891875  9194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:14.902062  9194 raft_consensus.cc:606] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:14.902406  9194 raft_consensus.cc:614] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:14.892998  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:14.904934  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:14.858966 13233 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:14.828070 13270 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:14.862771 13111 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:14.907224  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:14.907622  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:14.929222  9856 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:14.909162  9858 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:14.932016  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:14.932617  9858 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:14.850932 13156 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:05:14.900182  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:14.899070  9862 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
W20260709 13:05:14.912791  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 130184us
W20260709 13:05:14.913906  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 220704us
W20260709 13:05:14.936291  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:14.937722  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:14.858027 13240 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:05:14.968034  9854 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:14.905020 13283 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.003s	sys 0.013s
I20260709 13:05:14.977608  9856 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:14.910591 13235 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:14.906783 13347 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:14.908807 13350 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:14.911772 13288 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:14.889210 13151 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:05:14.884929 13313 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 13:05:14.856841 13153 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:15.001317 13285 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:14.904173 13342 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:05:14.984748 13269 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:15.014506  9858 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:14.904525 13230 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 13:05:14.976943 13283 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.003s	sys 0.014s
W20260709 13:05:15.019778  9087 leader_election.cc:336] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.565s (SENT)
I20260709 13:05:15.135787  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:15.136898  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 117179us
W20260709 13:05:15.137688  9087 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.567s (SENT)
I20260709 13:05:14.967356 13111 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:14.910470 13263 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:05:15.142067  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1015) took 1208 ms. Trace:
I20260709 13:05:15.151718  9854 rpcz_store.cc:276] 0709 13:05:13.933587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.937407 (+  3820us) service_pool.cc:224] Handling call
0709 13:05:13.937725 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:14.971469 (+1033744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:15.141796 (+170327us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400671e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:15.142002 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2369120000}
I20260709 13:05:15.162920  9269 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:15.018741 13350 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:14.853929 13299 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.189167  9189 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:15.211916  9189 raft_consensus.cc:606] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:15.212299  9189 raft_consensus.cc:614] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:15.041608 13285 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:15.187954  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 50276.7us
I20260709 13:05:15.168097 13276 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.179035 13288 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:15.148692 13343 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:15.222585 13357 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:05:15.129899 13207 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:05:15.180214 12800 tablet_bootstrap.cc:700] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Time spent opening tablet: real 0.140s	user 0.003s	sys 0.003s
I20260709 13:05:15.215704  9264 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
W20260709 13:05:15.216894  9087 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.652s (SENT)
I20260709 13:05:15.237713  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:15.238983  9087 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.716s (SENT)
I20260709 13:05:15.239436  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:15.224968 13354 log.cc:927] Time spent T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.098s	user 0.001s	sys 0.001s
W20260709 13:05:15.240737  9087 leader_election.cc:336] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.148s (SENT)
W20260709 13:05:15.241256  9826 leader_election.cc:336] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.086s (SENT)
W20260709 13:05:15.242033  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 304426us
I20260709 13:05:15.245896  9826 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:15.195292  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 313890us
I20260709 13:05:15.251148  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.252096 13362 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:15.252077  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:15.253048  9118 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:15.228482 13278 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.232842 13299 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:15.243988 13356 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 13:05:15.242125 12953 consensus_meta.cc:287] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.002s
I20260709 13:05:15.254899  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:15.246591 13269 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.247434 13240 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:15.279991 13270 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.227208 13184 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.280448 13171 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.249871 13378 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:15.247538 13111 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.283279 13122 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:15.283696 13122 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:15.283842 13269 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:15.284636 13122 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.268489 13288 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:15.285625 13122 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:15.285997 13269 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:15.288136 13364 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:05:15.289041 13376 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:05:15.293742 13353 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:05:15.295054  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1169) took 1523 ms. Trace:
I20260709 13:05:15.295436  9859 rpcz_store.cc:276] 0709 13:05:13.771929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:13.772041 (+   112us) service_pool.cc:224] Handling call
0709 13:05:13.772345 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:13.772684 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:13.772711 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:13.805370 (+ 32659us) log.cc:844] Serialized 26 byte log entry
0709 13:05:14.569120 (+763750us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b18003fbd40. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:05:14.811264 (+242144us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:14.811303 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:14.811366 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:15.294755 (+483389us) raft_consensus.cc:1685] finished
0709 13:05:15.294778 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:15.294994 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6704,"raft.queue_time_us":264,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":20696,"spinlock_wait_cycles":5259136,"thread_start_us":241955,"threads_started":1,"wal-append.queue_time_us":1228095}
I20260709 13:05:15.247694 13276 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:05:15.234465 13298 consensus_queue.cc:1175] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ce51b61da22f40d4aa0204a6b1028f28" responder_term: 12 status { last_received { term: 9 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: [11.9-11.9]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:15.195766 13347 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.299990  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:15.366673  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 73480.7us
I20260709 13:05:15.367079  9859 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:15.367758  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:15.368887  9114 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:15.369509  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:15.301252  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:15.315914  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.316874  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 62209.9us
I20260709 13:05:15.371474  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:15.373037  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.376533  9198 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:15.376897  9121 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:05:15.361008 13357 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:05:15.378614  9194 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:05:15.380123  9198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:15.380767  9198 raft_consensus.cc:606] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.382170  9089 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:15.383279  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:15.360116 13152 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:15.392799  9089 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:15.392942  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:15.397119  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:15.398213  9189 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:05:15.398593  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.372349 13002 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.014s	sys 0.012s
I20260709 13:05:15.368677 13288 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:05:15.404613  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.411453  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:05:15.411917  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:15.386310 13324 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.046s	user 0.019s	sys 0.000s
I20260709 13:05:15.390846 13341 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:15.385780 13276 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.416340  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.424789  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:15.424820  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.419144  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:15.385972 13355 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:15.427716 13152 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:15.428483 13270 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:15.428894 13270 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:15.429255 13270 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.429606  9192 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:15.434237  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.435554 13152 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:15.436067 13152 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.436139 13345 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:15.436525 13152 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:15.440346 13366 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:05:15.440781 12953 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:15.442358 12953 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:15.442799 12953 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:15.386135 13278 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
W20260709 13:05:15.435730 13368 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:05:15.444900 13276 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.447158 13276 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.448000 13361 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:15.448491  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.448625 13285 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.448923 13361 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:15.452661 13361 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:15.453099 13361 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:15.453554 13361 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.402904 13286 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.419577 13324 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.455631 13332 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:05:15.457441 13324 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.457798 13345 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:15.458093 13288 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.458763 13345 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.460636 13341 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:15.460945  9195 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:15.415380 13362 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.461779 13269 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:15.461515 13341 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.461920 13152 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.405258 13242 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.059s
W20260709 13:05:15.464378 13232 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.000s
W20260709 13:05:15.403266 13002 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.014s	sys 0.012s
I20260709 13:05:15.466428 13002 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:15.464454 13152 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.467566 13002 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:15.471577  9087 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.382s (SENT)
I20260709 13:05:15.472065  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:15.459302 13361 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.280105 13350 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.476459  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:15.481184  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:15.483422  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:15.509871  9122 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
W20260709 13:05:15.484814  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 50652.1us
I20260709 13:05:15.480978  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:15.511421  9859 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:15.513770  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.514380  9266 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:15.514520  9235 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:15.515499  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:15.515735  9263 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:15.516485  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.517995  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.520793  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.522934  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.524225  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:15.430992 13282 consensus_meta.cc:287] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.538s	user 0.006s	sys 0.004s
W20260709 13:05:15.522786  9826 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.547s (SENT)
I20260709 13:05:15.528239  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.528774  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.529649  9271 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:15.543949  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.557871  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.558842  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.560297  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.507393 13235 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:15.484239 13152 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:15.467667 13270 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.402719 13284 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:05:15.530280 13278 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:15.557798 13263 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:15.489740 13338 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:15.557049 13277 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:15.566008  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.565186  9264 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:15.566891  9265 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.572309  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.564931  9267 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:15.574041  9860 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:15.574808  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.575982  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:15.608577  9271 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:15.577492  9269 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:15.578959  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:15.580694  9859 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:15.584743  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.526193 13289 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:05:15.575369 13361 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:15.461592 13362 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.587954 13277 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.592644 13184 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.593146  9164 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.593451 13332 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.598652 13270 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.598922 13286 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:15.600442  9272 raft_consensus.cc:2410] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:05:15.609757  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:15.686949  9862 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:15.609462  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:15.612273  9826 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.633s (SENT)
W20260709 13:05:15.613144  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 194081us
W20260709 13:05:15.688241  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 76005.2us
I20260709 13:05:15.688563  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.689263  9271 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:15.689965  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.690166  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:15.690831 13378 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.618770  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.691293 13378 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.692778  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.693514 13378 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:15.694177 13263 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:15.694607  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.571938 13355 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:15.574280 13278 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.696244  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.696549 13278 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:15.697150  9271 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:15.699054  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.621582  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.626026  9238 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.564282 13282 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:05:15.700464  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 74586.2us
I20260709 13:05:15.700790  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.701156  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:15.701517 13282 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:15.702082 13282 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:15.702186  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.483271 12953 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.702728 13282 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:15.702828  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:15.665087  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.704222  9266 raft_consensus.cc:2368] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:15.704663  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:15.687479 13332 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:15.705529 13152 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:15.681731  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 88635.6us
I20260709 13:05:15.706786  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.707860  9196 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:05:15.708032  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.709998 13152 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:15.710096 13282 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:15.693785  9268 raft_consensus.cc:2410] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:05:15.711221  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.712419 13282 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:15.696017 13355 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.714478  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.715199  9271 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:15.712343  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.622735 13361 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:15.628137 13122 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.725035  9268 raft_consensus.cc:2393] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:15.722836  9270 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:15.726167  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.726511  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:15.733848  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.686797 13151 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:15.727337 13361 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:15.681303 13184 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.734645 13002 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:15.823228  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:15.828091  9857 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:15.783916  9267 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:15.815862  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:15.745139  9271 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:15.789549  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.829200  9861 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:15.829762  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.830328  9271 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:05:15.830363  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.822866  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:15.831943  9860 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:15.809247  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:15.784309  9265 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:15.833894  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.820869  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:15.834496  9265 raft_consensus.cc:2376] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9. Re-sending same reply.
I20260709 13:05:15.834806  9862 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
W20260709 13:05:15.775802 13360 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
W20260709 13:05:15.767969 13201 consensus_meta.cc:287] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.912s	user 0.004s	sys 0.008s
W20260709 13:05:15.784865 13359 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:05:15.741632 13355 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.780915 12979 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:15.836776 13387 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:05:15.871609  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.872510  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:15.873898  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:15.962734  9271 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
W20260709 13:05:15.962990  9826 leader_election.cc:336] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.851s (SENT)
I20260709 13:05:15.963479  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:15.963938  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.816361 13288 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.745500 13151 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:15.736245 13122 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.882651 13201 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:15.905658  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:15.916883  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 312166us
I20260709 13:05:15.772404 13372 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:15.972430 13326 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:15.959875 13002 raft_consensus.cc:697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:15.738816 13263 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 13:05:15.960479  9861 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:15.793058 13156 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:15.964519  9271 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:15.970221 13184 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.973246  9238 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:15.979113  9272 raft_consensus.cc:2410] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:05:15.975445 13298 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:15.954165 13233 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:05:15.980067  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.000077 13233 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:15.975901 13386 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.240s	user 0.002s	sys 0.007s
I20260709 13:05:15.998967 13349 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:05:16.000754  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:15.984512 13326 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.987946 13156 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:16.001998  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.002730 13151 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.003351 13371 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:15.987495 13362 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:15.976636 13288 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.004217 13156 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.004312 13233 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:15.984638  9268 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:15.986402  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:15.773625 13334 raft_consensus.cc:3055] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:15.772660 13383 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:16.005712 13332 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.005694 13233 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:16.005970 13288 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:16.008570  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:16.006264 13385 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.286s	user 0.003s	sys 0.006s
I20260709 13:05:16.008216 13334 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:16.016392 13334 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.017174 13334 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:15.977679 13278 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.021112 13122 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.013489 13263 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:16.028738 13278 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.030682 13326 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.031565 13230 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:16.032128 13230 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:16.031548 13361 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.035610 13361 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:16.086966  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:15.982599  9194 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:16.102165  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:16.106072  9087 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:16.108645  9264 raft_consensus.cc:2368] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10.
I20260709 13:05:16.109553  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:16.007279  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.008704  9272 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:16.114296  9271 raft_consensus.cc:2410] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:05:16.115037  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:16.115159  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.115902  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.117481  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.117830  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:16.118206  9271 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:05:16.118582  9272 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:16.119287  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:16.116611  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:16.111217  9265 raft_consensus.cc:2410] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:05:16.123463  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:16.125340  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:05:16.126129  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 112889us
I20260709 13:05:16.131806 13288 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:16.133477 13002 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.140540 13361 raft_consensus.cc:2764] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:05:16.141212 13276 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:16.141776 13383 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:16.141945 13276 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:16.142364 13383 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.033098 13230 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.140734 13362 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:16.147320 13341 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:16.150826 13276 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.152420 13341 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.143631 13383 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:16.140437 13338 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:16.139439 13284 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:16.156193 13341 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:16.160123 13355 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.161124 13355 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:16.162724 13285 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:16.154711 13122 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:05:16.165131 13355 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.144718 13362 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.165818 13288 raft_consensus.cc:3055] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:16.166270 13362 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:16.166270 13288 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:16.158470 13111 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:16.167923 13288 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.169835 13288 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:16.163527 13285 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.175697 13111 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:05:16.186743  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:16.191783  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:16.188513  9190 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:16.193320 13233 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:16.193915 13233 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.194438 13233 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:16.208778 13341 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.217159  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:16.217952 13285 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.218984  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.221395  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.235966  9267 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:16.227298  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2557) took 1443 ms. Trace:
I20260709 13:05:16.224079 12833 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.244469 12833 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.211548 13341 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.248493 12833 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.236786  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.250036  9115 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:16.250427  9268 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:05:16.252197  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:16.253340  9267 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:16.261193  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.220824 13285 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.264045 13361 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:16.264362  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 69578.7us
I20260709 13:05:16.264860 13153 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.265509  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:16.265676 13153 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.267791 13111 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:16.264469 13371 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.270794  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:16.290580  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:16.306931  9118 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:16.313114  9263 raft_consensus.cc:2410] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:05:16.319010 13285 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.319631  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.319862 13285 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.320569  9859 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:16.321445 13230 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:16.322433  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54861us
I20260709 13:05:16.349697  9087 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:16.324954  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.327957 13361 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.297353 13153 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
W20260709 13:05:16.334777  9190 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2099) took 2068 ms (client timeout 2060 ms). Trace:
I20260709 13:05:16.337781  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:16.370631  9190 rpcz_store.cc:269] 0709 13:05:14.265829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.330264 (+ 64435us) service_pool.cc:224] Handling call
0709 13:05:16.334412 (+2004148us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400151154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:16.334608 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4021100928}
I20260709 13:05:16.338246  9263 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:16.238267  9270 rpcz_store.cc:276] 0709 13:05:14.784005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.784190 (+   185us) service_pool.cc:224] Handling call
0709 13:05:14.784557 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:15.726384 (+941827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.226653 (+500269us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54002a4c54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:16.227049 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2154043136}
I20260709 13:05:16.378293  9115 raft_consensus.cc:2410] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:05:16.377117 13383 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.385902 13383 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.388120  9118 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:16.369347 13298 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:16.390918  9265 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:16.394971  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 45458us
I20260709 13:05:16.396786 13195 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:16.401389  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:16.402215  9270 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:05:16.402228  9855 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:16.389385 13180 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.377300 13288 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:05:16.416072 13336 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:16.406796 13195 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:16.417059 13336 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:16.417827 13195 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.418079 13336 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:16.420192 13343 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:16.420063 13331 consensus_queue.cc:1059] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:16.420717 13343 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:16.421650 13331 raft_consensus.cc:3055] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:16.422051 13331 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:16.421918 13343 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.423947 13355 raft_consensus.cc:3055] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:16.424408 13355 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:16.425436 13355 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:16.426901  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.427287  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:16.376729 13002 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:05:16.433708 13195 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.342846  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:16.435283  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 92606.1us
I20260709 13:05:16.435958  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:16.436213  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:05:16.437373  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54365.2us
I20260709 13:05:16.323515 13263 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:16.439946 13383 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:16.440222 13263 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:16.349897  9198 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 13:05:16.440734 13383 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:16.441043  9198 pending_rounds.cc:85] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 11
I20260709 13:05:16.441294 13383 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:16.406167 13298 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.442549 13383 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.350498  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:16.351300  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.444068 13270 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:16.444471 13298 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.354447  9271 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:16.353399 13346 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.445324 13270 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:16.356786 13156 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:16.445586 13346 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.446220 13180 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.448499 13180 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.358068  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2342) took 1088 ms. Trace:
I20260709 13:05:16.322696 13285 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.367651 13361 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:16.361763 13265 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.452397 13265 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.453490  9118 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:16.454636 13265 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:16.455085  9121 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:05:16.455577  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:16.456606  9114 rpcz_store.cc:276] 0709 13:05:15.286407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.298025 (+ 11618us) service_pool.cc:224] Handling call
0709 13:05:15.298370 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:15.369351 (+ 70981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.357509 (+988158us) spinlock_profiling.cc:243] Waited 70.3 ms on lock 0x7b5400254254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:16.357896 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161598208}
I20260709 13:05:16.459700  9123 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:16.444085  9122 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:16.369253 13283 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.433859 13341 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:16.435601 13362 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.322461  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.441403  9198 pending_rounds.cc:107] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260709 13:05:16.444591  9859 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:16.466722  9235 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:16.431408 13349 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.270182 13371 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.356057  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1197) took 1444 ms. Trace:
I20260709 13:05:16.445735 13270 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:16.341060 13233 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.472610  9198 raft_consensus.cc:2889] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:16.423736 13343 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:16.488364  9855 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:16.426901 13355 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:16.515764  9271 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:16.510023  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:16.528256  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 61617.2us
I20260709 13:05:16.489838 13346 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.474228  9853 rpcz_store.cc:276] 0709 13:05:14.910995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.911204 (+   209us) service_pool.cc:224] Handling call
0709 13:05:14.911507 (+   303us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:15.000867 (+ 89360us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:15.002036 (+  1169us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:16.232509 (+1230473us) spinlock_profiling.cc:243] Waited 6.65 ms on lock 0x7b2c00047988. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:05:16.233383 (+   874us) log.cc:844] Serialized 52 byte log entry
0709 13:05:16.282080 (+ 48697us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:16.282217 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.301267 (+ 19050us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.355473 (+ 54206us) raft_consensus.cc:1685] finished
0709 13:05:16.355500 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.355785 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22623,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":8353,"spinlock_wait_cycles":15289600,"thread_start_us":131156,"threads_started":2,"wal-append.queue_time_us":119695}
I20260709 13:05:16.423552 13331 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.537083  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:16.550966  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.551280  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.557484  9857 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:16.558257  9123 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:16.558166  9115 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: [8.7-9.8]
I20260709 13:05:16.562649  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:16.570024  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.570896 13361 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:16.571007 13002 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.571614 13361 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.572120 13361 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:16.571921 13002 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.574251 13002 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.520733 13232 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 13:05:16.588444  9861 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:16.553076 13170 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:05:16.590915  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:16.594178  9856 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:16.593073  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:16.595799 13265 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:16.599068 13153 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:16.608237  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 71259.6us
I20260709 13:05:16.608530  9862 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:16.609163  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 750) took 1185 ms. Trace:
I20260709 13:05:16.609555  9194 rpcz_store.cc:276] 0709 13:05:15.424072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.424933 (+   861us) service_pool.cc:224] Handling call
0709 13:05:15.425215 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.092350 (+667135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.608764 (+516414us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54003bd154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:16.608975 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1281496960}
I20260709 13:05:16.552132  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.530476 13341 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.610806  9196 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:16.549068 13283 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:16.552580  9859 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:16.612551 13341 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.613960 13283 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:16.614404 13371 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:16.578034 13270 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.552322 13257 raft_consensus.cc:3055] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:05:16.583832  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.585325 13233 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.578604  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:16.615798  9270 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:05:16.622706  9858 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:16.587519  9116 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:05:16.595960  9857 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:16.610119  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.583271  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1979) took 1043 ms. Trace:
I20260709 13:05:16.563455  9272 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:16.629583  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1990) took 1059 ms. Trace:
I20260709 13:05:16.623653  9264 rpcz_store.cc:276] 0709 13:05:15.539650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.619140 (+ 79490us) service_pool.cc:224] Handling call
0709 13:05:16.583009 (+963869us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54002a3354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:16.583170 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928286464}
I20260709 13:05:16.535317 13111 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.521384 13349 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.620491 13257 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:16.670576  9122 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:16.668488  9860 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
W20260709 13:05:16.660853  9087 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.587s (SENT)
I20260709 13:05:16.674434  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:16.674993  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.581825 13277 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.632436  9267 rpcz_store.cc:276] 0709 13:05:15.569666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.254203 (+684537us) service_pool.cc:224] Handling call
0709 13:05:16.254613 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.263029 (+  8416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.263063 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:16.263685 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:05:16.474211 (+210526us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:16.510511 (+ 36300us) log.cc:844] Serialized 14 byte log entry
0709 13:05:16.570351 (+ 59840us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b18003bbc00. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f7653571c0a 00007f765ccb1219 00007f76535f9a40 00007f76535f44e4 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287 00007f76566ff208
0709 13:05:16.570573 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.570654 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:16.616895 (+ 46241us) raft_consensus.cc:1685] finished
0709 13:05:16.625327 (+  8432us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.629520 (+  4193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30825,"prepare.queue_time_us":38120,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":2353,"raft.queue_time_us":70981,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":190,"spinlock_wait_cycles":4497664,"thread_start_us":204065,"threads_started":1,"wal-append.queue_time_us":221327}
I20260709 13:05:16.682332  9854 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:16.666702 13170 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:16.684342  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:16.666891 13284 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:16.685045  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:16.688010  9121 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:05:16.692988  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.693853 13284 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.694180 13334 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:05:16.694957  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:16.695670  9118 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:16.696341 13284 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.697461 13257 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.698650 13257 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:16.661958 13265 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:05:16.701692  9825 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:16.702514  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 875) took 1163 ms. Trace:
I20260709 13:05:16.702931  9269 rpcz_store.cc:276] 0709 13:05:15.520286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.520501 (+   215us) service_pool.cc:224] Handling call
0709 13:05:16.668689 (+1148188us) spinlock_profiling.cc:243] Waited 55.7 ms on lock 0x7b5400062a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:16.683256 (+ 14567us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128190336}
I20260709 13:05:16.704385  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.705206  9269 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:16.706652  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.706488 13170 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.708652  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:16.709383 13170 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.710158  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:16.712601  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 121757us
I20260709 13:05:16.715325  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:16.715884  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.716853  9267 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:16.720376  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.721082  9861 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:16.692291  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.722059  9861 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:16.722235  9264 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:05:16.724359 13362 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.724393  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.676090 13111 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.725240  9264 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:16.725716 12800 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:05:16.725828  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.726902 12800 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:05:16.726936 13111 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:16.728615 13230 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.729159  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.729213 13230 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.730034  9862 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:16.734961  9859 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:16.735865 13230 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.737334  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2569) took 1043 ms. Trace:
I20260709 13:05:16.734858  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:16.748420  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.737809  9271 rpcz_store.cc:276] 0709 13:05:15.693641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.445570 (+751929us) service_pool.cc:224] Handling call
0709 13:05:16.445904 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.521322 (+ 75418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.737104 (+215782us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b54000ebfd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:16.737277 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155184512}
I20260709 13:05:16.750288  9122 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:05:16.754017  9858 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
W20260709 13:05:16.755996  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 83416.5us
I20260709 13:05:16.760854  9272 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
W20260709 13:05:16.777114  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 119784us
I20260709 13:05:16.775900  9121 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:16.749110 13334 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:16.782169  9263 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:16.753298 13336 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:16.755921 13284 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:16.790527  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.777050 13233 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:16.839005 13270 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:16.839257 13233 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.839699 13233 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:16.801036  9266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 1967) took 1263 ms (client timeout 1567 ms). Trace:
W20260709 13:05:16.843907  9266 rpcz_store.cc:269] 0709 13:05:15.537133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:15.584502 (+ 47369us) service_pool.cc:224] Handling call
0709 13:05:16.800617 (+1216115us) spinlock_profiling.cc:243] Waited 93.6 ms on lock 0x7b54002a3d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:16.800857 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215251328}
I20260709 13:05:16.802632  9856 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:16.803258  9268 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:16.846735 13331 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:16.778257 13362 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.848068 13362 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.850082  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2574) took 1109 ms. Trace:
I20260709 13:05:16.850250 13362 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.850517  9263 rpcz_store.cc:276] 0709 13:05:15.741159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.602300 (+861141us) service_pool.cc:224] Handling call
0709 13:05:16.761516 (+159216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.787687 (+ 26171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.849881 (+ 62194us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54001c61d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:16.850023 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365384960}
I20260709 13:05:16.851209  9267 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:05:16.851603  9267 pending_rounds.cc:85] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 9
I20260709 13:05:16.851897  9267 pending_rounds.cc:107] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:05:16.852157  9267 raft_consensus.cc:2889] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:16.804666  9853 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 6 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:05:16.807964  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.812171  9265 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:16.854899  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2573) took 1123 ms. Trace:
I20260709 13:05:16.815828 13283 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.791225 13180 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:16.856115 13283 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.856258 13180 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.857191 13156 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:16.858354 13156 raft_consensus.cc:3055] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:05:16.825254  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 113564us
I20260709 13:05:16.859504  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 1988) took 1290 ms. Trace:
I20260709 13:05:16.859894  9268 rpcz_store.cc:276] 0709 13:05:15.569213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.251240 (+682027us) service_pool.cc:224] Handling call
0709 13:05:16.803002 (+551762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.846783 (+ 43781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.859294 (+ 12511us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54002a3d50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:16.859428 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1268047232}
I20260709 13:05:16.861579  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.862249 13156 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:16.829263  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.863797 13156 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.864552  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:16.865098 13156 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:16.786728 12800 ts_tablet_manager.cc:1403] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 2.237s	user 0.042s	sys 0.031s
I20260709 13:05:16.828402 13171 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:16.784426 13346 consensus_queue.cc:1059] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:05:16.872737  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:16.873879 13346 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:16.874265 13346 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:16.874272  9195 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:16.807448  9270 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:16.858551 13180 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.880374  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.832814  9271 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:16.834997 13413 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:16.875272 13346 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.884825 13413 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.885344 13346 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:16.836088  9235 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:16.837061  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:16.887084  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.837157  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:16.837731 13361 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.782725 13334 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:16.838704  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2575) took 1067 ms. Trace:
I20260709 13:05:16.893188  9269 rpcz_store.cc:276] 0709 13:05:15.771393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.752077 (+980684us) service_pool.cc:224] Handling call
0709 13:05:16.788074 (+ 35997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.838338 (+ 50264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.838379 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:16.838435 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:16.838470 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.838527 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:16.838635 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:16.845698 13349 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:16.819895 13345 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:16.858433 13283 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:16.861500  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:16.865175  9196 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:16.830897 13289 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:05:16.830148  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 128554us
I20260709 13:05:16.865729 13151 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.855254  9265 rpcz_store.cc:276] 0709 13:05:15.731661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.578667 (+847006us) service_pool.cc:224] Handling call
0709 13:05:16.811833 (+233166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:16.854153 (+ 42320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:16.854732 (+   579us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54000ea6d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:16.854843 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535424768}
I20260709 13:05:16.867659  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:16.869628  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.869532  9021 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 8 to 9, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 9 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:16.872702 12800 raft_consensus.cc:359] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 10 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:05:16.895520  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 106193us
W20260709 13:05:16.898195  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 61129.6us
I20260709 13:05:16.911823  9267 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:16.916908  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.898255 13170 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:16.919361  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:16.919700 13170 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.922070 13170 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:16.922689  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:16.922586 13170 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:16.923660 13170 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.924067 13201 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:16.924376 13151 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:16.938671 13289 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:16.924394  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:16.939637  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.940503  9197 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:16.938762  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:16.941679  9123 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:16.942117  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.925684  9190 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:16.906842  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:16.910013  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:05:16.938099  9020 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36626 (request call id 359) took 29298 ms (client timeout 15000 ms). Trace:
I20260709 13:05:16.938480  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:16.915302  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:16.953735  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 47069us
I20260709 13:05:16.954495  9089 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:16.955469  9195 raft_consensus.cc:2410] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:16.952458  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:16.958048  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:16.958168  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.958407  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.959033  9197 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:16.960332  9197 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:05:16.959878  9021 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 5 to 6, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 6 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:16.961210  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:16.961844  9190 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:16.961609  9021 catalog_manager.cc:5697] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 6 to 7, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 7 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:05:16.964552  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:16.964339 13334 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.965827 13334 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:16.968916 13413 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.973142 13361 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:05:16.973348  9190 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
W20260709 13:05:16.943795  9020 rpcz_store.cc:269] 0709 13:04:47.639375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:47.651903 (+ 12528us) service_pool.cc:224] Handling call
0709 13:05:16.937807 (+29285904us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:13.484561 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:13.485101 (+   540us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:13.485148 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:05:13.485166 (+    18us) tablet.cc:662] Decoding operations
0709 13:05:13.486165 (+   999us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:13.486270 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:13.486309 (+    39us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:05:13.487285 (+   976us) tablet.cc:701] Row locks acquired
0709 13:05:13.487333 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:05:13.487504 (+   171us) write_op.cc:270] Start()
0709 13:05:13.487641 (+   137us) write_op.cc:276] Timestamp: P: 1783602313487440 usec, L: 0
0709 13:05:13.487658 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:05:13.502102 (+ 14444us) log.cc:844] Serialized 3908 byte log entry
0709 13:05:13.683959 (+181857us) op_driver.cc:464] REPLICATION: finished
0709 13:05:13.798109 (+114150us) write_op.cc:301] APPLY: starting
0709 13:05:13.798248 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:14.323155 (+524907us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:14.323284 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:14.338586 (+ 15302us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:14.338898 (+   312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:05:14.338941 (+    43us) write_op.cc:312] APPLY: finished
0709 13:05:14.339262 (+   321us) log.cc:844] Serialized 89 byte log entry
0709 13:05:14.339442 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:14.339564 (+   122us) write_op.cc:454] Released schema lock
0709 13:05:14.339620 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:14.421325 (+ 81705us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:14.423996 (+  2671us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2536063,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":113805,"apply.run_cpu_time_us":19446,"apply.run_wall_time_us":602423,"cfile_cache_hit":34,"cfile_cache_hit_bytes":35547,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":1,"lbm_read_time_us":6833,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":7,"num_ops":9,"prepare.queue_time_us":13222,"prepare.run_cpu_time_us":4450,"prepare.run_wall_time_us":58556,"replication_time_us":192443,"thread_start_us":82269,"threads_started":2}]]}
I20260709 13:05:16.902205 13371 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:16.975871  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.976614  9189 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:16.963140  9021 catalog_manager.cc:5697] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 4 to 6, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 6 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:05:16.956228  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.940716 13151 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:16.909075 13362 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:16.951467  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:16.979473  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:16.943048 13170 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:16.982015  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.927426 12800 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 10 LEARNER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: LEARNER
I20260709 13:05:16.977104 13122 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:16.990348 13122 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:16.990147 12800 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:16.991290 13122 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.992162 13232 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:16.992614 12800 ts_tablet_manager.cc:1434] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.127s	user 0.007s	sys 0.001s
I20260709 13:05:16.986196 13361 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:16.994535 13361 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:16.995026 13361 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:16.971881 13345 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:16.996717 13345 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:16.998337 13345 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:16.977911  9198 raft_consensus.cc:2376] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10. Re-sending same reply.
I20260709 13:05:16.993861 13375 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:16.989075  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:16.990172  9194 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:16.986541  9114 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:05:17.001057  9192 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:17.002115 13375 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:17.012567  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:17.015924  9235 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:17.018306  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:17.019150  9198 raft_consensus.cc:2376] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7. Re-sending same reply.
W20260709 13:05:17.019244  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 64875.1us
I20260709 13:05:17.002342 13413 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:17.010792 13362 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:16.999234  9021 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 3 to 6, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 9 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:17.045239  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.049602  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:17.048449  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 110110us
I20260709 13:05:17.047274  9860 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 10 LEARNER]: Advancing to term 11
I20260709 13:05:17.053846  9087 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:17.055073  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:17.056336  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:17.058542  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:17.062810 13182 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:17.063527 13337 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:17.063391 13362 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:17.063475 13182 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.062704 13349 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:17.066391 13349 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.066731 13349 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:17.063472  9021 catalog_manager.cc:5697] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:05:17.069489  9021 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 4 to 11, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 11 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:17.026341 13343 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:05:17.095393  9856 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:17.062647 13284 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:17.035243 13180 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:05:17.100051  9853 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:17.058302 13289 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:17.059041 13283 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:17.078408  9021 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 8 to 10, now has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:17.104050  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:17.108987  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.105198 13284 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:17.114638 13284 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:17.117825 13337 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:17.120153 13372 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:05:17.121770  9021 catalog_manager.cc:5697] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 8, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 8 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:05:17.117985 13182 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:17.123859 13274 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:05:17.126672  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:17.128361  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:17.131103 13424 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:05:17.107430 13289 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:17.132151 13289 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:17.136876  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.138078  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.138629  9190 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:17.147686 13285 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.148330 13285 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.149572  9123 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:05:17.150854  9190 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:17.150770  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:17.154922  9194 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:17.123225  9021 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:05:17.168881  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56869us
I20260709 13:05:17.172009  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:17.175448  9196 raft_consensus.cc:2410] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:05:17.176931  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:17.179401  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 131403us
I20260709 13:05:17.181236  9198 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:17.181435  9087 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:17.184110  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:17.184616  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:17.183466  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.187131  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:17.187551 13286 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.188292  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:17.191063  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.204113  9265 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:17.188306 13286 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:17.213297  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:17.214428  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:17.080372 13375 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:17.216812 13375 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:17.217337 13423 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:05:17.217754 13413 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:05:17.221096  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 899) took 1265 ms. Trace:
I20260709 13:05:17.221469  9264 rpcz_store.cc:276] 0709 13:05:15.955713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:16.725906 (+770193us) service_pool.cc:224] Handling call
0709 13:05:16.726231 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:16.737014 (+ 10783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:16.737053 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:16.737641 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:05:17.034404 (+296763us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:17.056603 (+ 22199us) log.cc:844] Serialized 14 byte log entry
0709 13:05:17.056879 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:17.056951 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:17.146550 (+ 89599us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b18003e2de0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:17.220837 (+ 74287us) raft_consensus.cc:1685] finished
0709 13:05:17.220927 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:17.221043 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23902,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":10124,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":2636,"spinlock_wait_cycles":50045184,"thread_start_us":296364,"threads_started":1,"wal-append.queue_time_us":363236}
I20260709 13:05:17.180007 13371 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:17.227068 13371 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:17.228674 13427 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:05:17.230613  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 121314us
I20260709 13:05:17.206115  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:17.148528 13283 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:17.179016 13156 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.209412  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:17.208981 13195 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:17.232017  9087 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:17.248131  9856 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:05:17.250258  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.233690  9193 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:17.189584  9269 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:17.245903 13343 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:17.223306 13349 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.253214  9021 catalog_manager.cc:5697] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 5 to 6, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 6 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.258991  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:17.254446 13151 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.246404 13283 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:17.180550 13233 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.278258  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:17.287065  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:17.287698  9190 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:17.288178  9194 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
W20260709 13:05:17.291697  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.246687 13195 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:17.296051  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:17.288589  9194 pending_rounds.cc:85] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 7
I20260709 13:05:17.287278  9116 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:17.296192  9192 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: [8.6-11.7]
I20260709 13:05:17.296697  9194 pending_rounds.cc:107] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:05:17.297031  9194 raft_consensus.cc:2889] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:17.254894 13337 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:17.298390  9021 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.212327 13289 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.301043  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:17.301532  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:17.263978 13343 raft_consensus.cc:697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:17.308940  9826 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:17.287876 13413 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:17.313670  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:17.314103  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.316061 13278 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:05:17.316471  9856 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:17.345392  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:17.316861  9281 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-685fa3ec557f4f3db02ec6ccce1d6f3e received from {username='slave'} at 127.0.0.1:38650
I20260709 13:05:17.321074  9855 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:17.313686 13274 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:17.321130 13233 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:17.372162  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:17.373730  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.374436  9192 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:17.323410  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:17.325060 13283 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.327224 13298 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:17.329272  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:17.329967  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:17.330250  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.330950 13230 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Committing config change with OpId 8.8: config changed from index 5 to 8, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:17.336949 13371 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:17.385505  9197 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:17.338693 13277 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.282409 13171 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:05:17.340720 13289 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:17.306028 13337 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:17.254791 13156 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.400467  9115 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:17.401633  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:17.402776 13156 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:17.307257 13170 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:17.403733 13277 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.405280  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.343569  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.406177  9193 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:17.299939 13195 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:17.360416  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.362690  9235 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:17.374488 13122 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.324153 13234 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:17.376238  9116 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:17.377230  9123 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:17.377332  9190 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:17.378048  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.345912 13276 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:17.413306  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 50720.5us
I20260709 13:05:17.398464 13313 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:17.399931  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:17.419807  9119 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:05:17.342453  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:17.406788  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:17.413415  9825 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:17.421078 13289 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:17.311128 13338 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:05:17.417152  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.416991  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:17.427032  9197 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:17.310948 13151 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:17.428006  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 98853.4us
W20260709 13:05:17.424736  9089 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.843s (SENT) [suppressed 41 similar messages]
I20260709 13:05:17.432220  9114 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:17.407599 13337 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:17.404903 13170 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:17.396567 13278 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:17.414721  9281 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-685fa3ec557f4f3db02ec6ccce1d6f3e on tablet 685fa3ec557f4f3db02ec6ccce1d6f3e with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:17.422746 13270 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:17.503798 13270 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.504226 13270 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:05:17.506664 13361 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260709 13:05:17.507102 13361 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.410724 13180 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:05:17.455848  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:17.514281 13361 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:17.437508  9120 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
W20260709 13:05:17.479863 13430 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:05:17.481408 13439 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:05:17.524451  9267 raft_consensus.cc:2368] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13.
I20260709 13:05:17.485749  9119 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
W20260709 13:05:17.485891 13346 consensus_meta.cc:287] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.000s
I20260709 13:05:17.438822 13234 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:17.527189 13234 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:17.527835  9191 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:17.439514 13122 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.488377  9021 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 8 to 9, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 9 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:17.532920  9021 catalog_manager.cc:5697] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 6, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 6 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:17.534372  9021 catalog_manager.cc:5697] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 6, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 6 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.492497 13283 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:17.493217 13334 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.002s
I20260709 13:05:17.463685 13343 consensus_queue.cc:237] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:17.536958  9195 raft_consensus.cc:2410] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:17.537791  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:17.538905  9119 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:05:17.498194 13277 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:17.502086 13286 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:17.502823  9198 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
W20260709 13:05:17.459982  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.508522  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:17.540182  9266 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:17.540616  9021 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.542240  9021 catalog_manager.cc:5697] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 8, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 8 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:17.509312  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:17.531452 13122 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:17.524724  9189 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:17.544056  9270 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:17.511495  9853 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:05:17.544909  9021 catalog_manager.cc:5697] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 2 to 7, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 7 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:17.546661  9021 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 9, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 9 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:17.551085  9266 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:17.557117 13338 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:17.558007 13338 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:17.560758  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:17.561545  9120 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:17.562932 13337 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:17.564373  9114 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:17.574432  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 149071us
I20260709 13:05:17.491537 13284 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.591388  9021 catalog_manager.cc:5697] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 3 to 4, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 4 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:17.629148  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 204420us
I20260709 13:05:17.629408  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:17.630434  9264 raft_consensus.cc:2368] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:05:17.636817 13341 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.639918 13263 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:17.657692 13445 log.cc:927] Time spent T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.177s	user 0.004s	sys 0.000s
I20260709 13:05:17.660775 13338 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:17.662525 13442 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:05:17.667086 13151 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:17.668761 13111 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.669350 13111 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.676450  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:17.687752 13341 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:17.711985  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 229583us
I20260709 13:05:17.712368 13195 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:17.712766  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:17.715070 13377 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:17.715551 13377 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:17.715951 13377 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:17.717334 13182 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:17.718521 13182 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:05:17.719759  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 265382us
W20260709 13:05:17.720160  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.723423  9117 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:05:17.724552  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48196.9us
I20260709 13:05:17.688488 13289 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:05:17.693382 13277 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:05:17.705946 13233 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:17.708509 13337 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:17.711565 13341 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:17.741497  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:17.742170  9119 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:17.746399  9087 leader_election.cc:400] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:05:17.746903  9087 leader_election.cc:403] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:17.747979  9161 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:17.678952 13263 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.763231 13263 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:17.764262  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:17.763980  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:17.771155  9854 raft_consensus.cc:2410] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:17.784935  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:17.785883  9853 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:05:17.791460  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.740303 13201 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:17.792125 13313 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:17.792521 13438 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.312s	user 0.002s	sys 0.000s
I20260709 13:05:17.791980 13452 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:05:17.805138  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:17.774893 13182 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:17.745662 13371 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:05:17.816516  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:17.780074 13286 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:17.813740  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:17.788452  9858 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Committing config change with OpId 8.8: config changed from index 5 to 8, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:17.817262  9859 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:17.820102  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:17.767664 13331 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 13:05:17.827037 13457 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:05:17.832233  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:17.844316  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:17.856017  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:17.857201  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 99843us
I20260709 13:05:17.873025  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:17.815236 13182 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:17.880741  9120 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:17.889366  9198 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:05:17.889866  9198 pending_rounds.cc:85] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 7
I20260709 13:05:17.890126  9198 pending_rounds.cc:107] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:05:17.890390  9198 raft_consensus.cc:2889] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:17.892827  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:17.873066 13283 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.916656  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:17.923534  9161 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:17.885237 13170 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:17.894063 13313 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:17.936862  9191 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:05:17.940070 13422 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:05:17.948024  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:17.886834 13232 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:05:17.908202 13195 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:17.972643  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:17.954442  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:17.984990  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:17.872720 13377 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.985688  9196 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:17.938788 13111 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:18.005559  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:18.009302  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 85886.5us
I20260709 13:05:17.926483 13283 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:17.945518 13230 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:17.945448 13170 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.016281  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:18.018110 13460 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:05:18.018329 13170 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:18.019506  9197 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:18.020980 13377 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:17.949523 13334 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:18.028120 13334 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.028687 13334 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:18.030304 13180 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.030908 13180 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:18.031872 13448 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.033068 13448 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.033619 13180 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:18.033771 13467 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.033536 13448 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.034484 13467 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.036441 13448 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:18.036916 13467 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:18.033506 13230 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:17.996111 13277 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:17.983819 13343 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.037988 13343 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:18.038046 13277 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.040051 13343 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:18.040948 13285 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:17.983388 13288 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:05:17.934427 13233 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:18.044595  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:18.044754  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:18.045786  9192 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:18.049278  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.040969 13277 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:18.050287  9120 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:17.983630 13362 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:05:18.053996  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.054193  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.055611  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.055361 13383 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:18.014951 13182 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.060093  9122 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:18.045958 13230 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:18.062597 13469 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:18.062872 13270 consensus_meta.cc:287] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.002s
I20260709 13:05:18.063133 13469 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.063333 13270 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.064541  9265 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:18.063970  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.064918 13283 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:18.065394 13469 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:18.065472 13270 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:18.065982 13383 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:18.061513  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.066942  9120 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:05:18.067529 13461 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:05:18.069593 13182 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:18.073340  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.074290 13383 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:18.075709  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:18.080525  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.081499  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.081871  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:18.082345  9114 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:18.081530  9118 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:05:18.077510  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:18.086146  9118 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Committing config change with OpId 8.8: config changed from index 5 to 8, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:18.087781  9825 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:18.089080 13277 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:18.089500 13277 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.089824 13277 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:18.092555 13441 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.609s	user 0.004s	sys 0.000s
I20260709 13:05:18.093752  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:18.094460  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:18.094470  9856 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 4.
I20260709 13:05:18.102139 13284 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:18.092618  9854 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:18.112596 13273 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:18.113112 13273 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:18.118532  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.119840 13283 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:18.123227  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.099937 13277 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.131618  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:18.124502  9120 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:18.122605  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:18.139503 13466 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:18.114074 13383 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:18.144327  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:18.120182 13288 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.151477  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2121) took 1101 ms. Trace:
I20260709 13:05:18.143029 13466 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.156297  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:18.153409  9195 rpcz_store.cc:276] 0709 13:05:17.050325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.050436 (+   111us) service_pool.cc:224] Handling call
0709 13:05:18.147333 (+1096897us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b5400217d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:18.151384 (+  4051us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097843584}
I20260709 13:05:18.160809  9192 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:18.161145  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:18.140028 13362 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:18.134830 13371 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:05:18.178761  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.179409  9194 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:18.180636  9195 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:18.182363  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.182648  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:18.183385  9122 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:18.170925  9161 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:18.171875 13462 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.171895  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.172104  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.160751 13466 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:18.173764  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:18.185357  9196 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:18.185557  9193 raft_consensus.cc:2376] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7. Re-sending same reply.
I20260709 13:05:18.163136  9190 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:05:18.173139 13274 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.139818 13273 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:18.162101  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.187441  9190 pending_rounds.cc:85] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 8
I20260709 13:05:18.188090  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.188592  9115 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:05:18.190735  9190 pending_rounds.cc:107] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:05:18.190927  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:18.191378  9190 raft_consensus.cc:2889] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:18.191409  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:18.191730  9192 raft_consensus.cc:2376] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12. Re-sending same reply.
I20260709 13:05:18.192633  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.192685  9198 raft_consensus.cc:2410] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:05:18.193539  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.140411 13171 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:05:18.198994  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.201782  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.199976  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:18.202713  9196 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:18.202857  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:18.203567  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:18.201655  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.205170  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.200655  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.206591 13478 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:18.206712  9195 raft_consensus.cc:2410] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:18.206943  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.207803 13478 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.184141  9189 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:18.234731  9189 raft_consensus.cc:606] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:18.235096  9189 raft_consensus.cc:614] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:18.210453 13469 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:18.208634 13285 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.238366  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.239384  9196 raft_consensus.cc:2368] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:05:18.239948  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.142603 13233 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.204886 13467 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.188613 13286 raft_consensus.cc:2764] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:05:18.238044  9193 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:18.248426 13270 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:18.248960 13448 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.248750 13361 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:18.267486  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:18.197621 13274 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.252815  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:18.257723  9856 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:18.197840 13462 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.242251 13362 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:18.256350 13273 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:18.288034  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:18.289572  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:18.291978  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:18.292752  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.292627  9270 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:05:18.294996  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:18.305117  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.323011  9114 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:18.297312  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:18.324828  9271 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:18.296954  9264 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:18.306030  9194 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:18.299135  9087 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:18.260195 13285 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:18.333797  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:18.284011 13201 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:18.291095 13278 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.297099  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:18.262041 13230 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.351537  9164 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.350312  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:18.291198 13338 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.298041 13383 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:18.334020 13452 consensus_meta.cc:287] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.000s
I20260709 13:05:18.263581 13448 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.361470 13232 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:18.362129  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.363087  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:18.363303 13448 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:18.344573 13156 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:18.345156 13171 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:18.363916 13475 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:18.363394  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.368825  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 902) took 1195 ms. Trace:
I20260709 13:05:18.369359  9267 rpcz_store.cc:276] 0709 13:05:17.173518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.185323 (+ 11805us) service_pool.cc:224] Handling call
0709 13:05:18.368565 (+1183242us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54000eb0d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:18.368747 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642638080}
I20260709 13:05:18.272966 13467 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.350734 13201 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.384639 13201 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:18.384722 13467 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:18.377461  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:18.401621  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 67886.3us
I20260709 13:05:18.403609  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.356230 13277 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:18.308740 13362 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:18.409988  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.337595 13377 consensus_queue.cc:1059] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:05:18.406095  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:18.413877  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 233486us
W20260709 13:05:18.414103  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 146692us
I20260709 13:05:18.377532 13361 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:18.363922 13156 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:18.365293 13452 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:05:18.468897  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:18.472342  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 66337.8us
W20260709 13:05:18.471428  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 134108us
I20260709 13:05:18.420881  9238 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.461180  9272 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Committing config change with OpId 8.8: config changed from index 5 to 8, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:18.359728 13278 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:18.558174  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:18.572058  9271 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:18.373090 13338 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.604619  9267 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:18.404676 13346 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:18.411648 13349 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:18.616268  9119 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:18.381146 13448 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.223s
I20260709 13:05:18.575624 13377 raft_consensus.cc:3055] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:18.410907 13465 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.550570 13171 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.472800 13298 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.513590 13156 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:18.612347 13480 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.371s	user 0.001s	sys 0.000s
I20260709 13:05:18.619611 13489 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:18.621126  9271 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:18.612318 13277 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.512871 13493 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:18.608906 13361 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.411653 13151 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:18.627931 13377 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:18.633345 13346 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:18.650965  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 230193us
I20260709 13:05:18.661185  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:18.665627 13285 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:18.666422 13285 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:18.671057  9264 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:05:18.671819  9267 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
W20260709 13:05:18.672334 13481 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:05:18.673697 13278 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:18.685065 13230 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:18.688166  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:18.689050  9855 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:18.689174 13375 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:18.689770 13375 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.690145  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:18.690971  9855 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:18.630172 13274 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:18.692271 13338 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:18.691259  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:18.690987  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:18.639822 13383 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:18.690322  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:18.459316 13471 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.198s
I20260709 13:05:18.652779  9089 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:18.650403 13493 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:18.719811  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 67264.7us
I20260709 13:05:18.720934  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:18.653975 13462 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:18.640821 13298 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.639133 13171 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:18.819711  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:18.704895 13346 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:18.705138 13375 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:18.812197 13484 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.468s	user 0.008s	sys 0.000s
W20260709 13:05:18.824936  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 220471us
I20260709 13:05:18.832280  9856 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:05:18.637437 13465 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.655810 13277 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.718304 13493 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:18.656689 13361 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.717581 13489 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.800269 13285 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:18.908915  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:18.913107  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 192290us
I20260709 13:05:18.818699 13500 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:18.812472 13448 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.920756  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:18.924103  9114 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:18.926136  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:18.930588  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:18.934490  9116 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:18.938539  9853 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:18.940438  9089 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:18.944864  9089 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:18.952199  9859 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:05:18.958719  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:18.915902 13285 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:18.974222  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:18.918217 13503 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:18.975283  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.980211  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:18.981098  9195 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:18.980389  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.935082 13338 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:18.984311  9190 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:18.991505  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.992378  9194 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:18.995108  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:18.958573 13274 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:18.986997  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:18.974524 13285 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:18.897855 13377 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.965106 13467 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:18.937598 13448 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.020787  9195 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:18.984642 13478 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:18.928002 13489 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:18.977844 13506 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:19.030224  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:19.022527 13182 consensus_meta.cc:287] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.952s	user 0.007s	sys 0.006s
I20260709 13:05:19.003049 13505 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:19.050179  9856 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:05:18.960587 13362 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:19.052945  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:19.029551 13201 consensus_meta.cc:287] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.004s
I20260709 13:05:19.000923 13372 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 13:05:19.057531  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 148709us
I20260709 13:05:19.033066 13503 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:19.030591 13471 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.035197 13494 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:19.042711 13478 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.064000 13448 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:19.064168 13451 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:19.064815  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.064595 13504 consensus_queue.cc:1059] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:05:19.019342 13274 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.057440 13336 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:19.002787 13383 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:19.083894 13498 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
W20260709 13:05:19.084044 13488 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
W20260709 13:05:19.084551 13491 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.399s	user 0.001s	sys 0.000s
I20260709 13:05:19.085045 13465 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:19.087829 13273 consensus_meta.cc:287] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.782s	user 0.005s	sys 0.006s
W20260709 13:05:19.088663 13496 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:05:19.088754 13504 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.089001 13156 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.089429 13504 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.091254 13156 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.035614 13467 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:19.075947  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 104790us
W20260709 13:05:19.065374 13466 consensus_meta.cc:287] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.864s	user 0.008s	sys 0.003s
I20260709 13:05:19.077162 13503 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.010669 13346 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.097165  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:19.066289 13362 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.070314 13201 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.078306 13506 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:19.053272 13462 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
W20260709 13:05:19.092324 13334 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.063s	user 0.001s	sys 0.012s
I20260709 13:05:19.104717 13156 raft_consensus.cc:3055] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:19.131395  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:19.096580 13448 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:19.078855 13451 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:19.143088  9114 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:19.144954  9115 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:19.124006  9089 leader_election.cc:400] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
W20260709 13:05:19.145852  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 157346us
I20260709 13:05:19.093986 13500 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:19.147171  9196 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:19.093246 13504 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.104955 13274 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:19.152897  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:19.152997  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:19.123620 13171 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.151770  9089 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:19.138342 13273 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:19.162528  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 501425us
I20260709 13:05:19.077335 13471 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.103497 13489 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.157658 13337 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:05:19.111147 13466 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.144320 13156 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:19.151108 13448 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.192164  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.161757 13180 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:19.147164 13331 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:19.157961  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:19.150110 13298 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.179404  9189 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:19.201905  9120 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:19.202258  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.179936  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.181540  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:19.182801 13489 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:19.184669 13478 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.184383 13500 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:19.187458  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:19.157809 13472 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:19.158497 13361 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.155133 13451 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:19.141368 13362 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.145349 13334 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.200821  9117 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:19.178325 13504 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:19.199508  9828 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:19.204859  9121 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:05:19.249482  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 50063.4us
I20260709 13:05:19.194216 13448 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:19.218513  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.257490  9190 raft_consensus.cc:2368] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:05:19.216473  9122 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:19.206660  9114 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:19.224007 13467 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.262439  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:19.220429 13201 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.232102 13277 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:19.271328  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 89856.8us
I20260709 13:05:19.228502 13503 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.282163  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:19.282793  9122 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:19.264721 13520 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:19.284652  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.284745 13520 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.285303 13520 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:19.285537  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:19.289896 13515 log.cc:927] Time spent T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Append to log took a long time: real 0.198s	user 0.002s	sys 0.000s
I20260709 13:05:19.292827 13334 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.278479  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:19.239691  9115 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:19.194555 13180 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.203920 13331 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:19.217013 13485 log.cc:927] Time spent T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.645s	user 0.002s	sys 0.000s
I20260709 13:05:19.142436 13506 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.246106  9269 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:05:19.224686 13517 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:19.206610 13372 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:19.297317  9117 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:19.226699 13156 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.296273  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:19.262600 13518 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:19.275767 13504 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:19.260949 13233 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:19.259433 13273 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:19.307303  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:19.311157  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.260721 13298 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.315047  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.274571 13283 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:19.317770  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.283582 13494 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:19.331449  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.298960 13180 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:19.312685 13270 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.049s	user 0.004s	sys 0.039s
I20260709 13:05:19.321456  9114 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:19.328706  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:19.345909  9122 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:19.328603  9115 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:19.340888  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.351040  9858 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
W20260709 13:05:19.359393 13513 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:05:19.360185 13504 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.360849  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2143) took 1053 ms. Trace:
I20260709 13:05:19.361248  9193 rpcz_store.cc:276] 0709 13:05:18.307644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.329300 (+ 21656us) service_pool.cc:224] Handling call
0709 13:05:18.350322 (+ 21022us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.357151 (+  6829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.357200 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.368304 (+ 11104us) log.cc:844] Serialized 26 byte log entry
0709 13:05:19.000730 (+632426us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:19.000793 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.000887 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.358679 (+357792us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:19.360513 (+  1834us) raft_consensus.cc:1685] finished
0709 13:05:19.360533 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.360788 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52163,"raft.queue_time_us":6796,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":19957,"spinlock_wait_cycles":10472192,"thread_start_us":491544,"threads_started":1,"wal-append.queue_time_us":686201}
I20260709 13:05:19.363245  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.366251 13493 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.320648 13372 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:19.368425 13500 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.368980 13500 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.371268 13500 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:19.376735  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:19.372031 13500 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.377823 13524 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.385133 13524 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.385103 13500 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.387486 13524 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.388924  9115 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:19.392310  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:19.395901 13499 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:19.396790 13499 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:19.397795 13519 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:05:19.398011 13499 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.401782 13331 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.403014 13503 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:19.403386 13377 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:19.403913 13503 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.405488  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:19.338675 13283 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:19.406836  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:19.408100 13528 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:19.409185  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1230) took 1289 ms. Trace:
I20260709 13:05:19.409703 13524 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:19.410506 13524 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:19.410833 13524 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.411182 13524 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:19.413524  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.414132  9264 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:05:19.342944 13270 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.004s	sys 0.040s
I20260709 13:05:19.419265  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.419921  9265 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:19.425534  9272 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:19.368638 13372 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.428509  9826 leader_election.cc:400] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13.
I20260709 13:05:19.429122  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.430284  9264 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:19.371886 13383 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.409718  9854 rpcz_store.cc:276] 0709 13:05:18.120234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.120394 (+   160us) service_pool.cc:224] Handling call
0709 13:05:18.120720 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:18.136545 (+ 15825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.136586 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:18.143434 (+  6848us) log.cc:844] Serialized 26 byte log entry
0709 13:05:18.272036 (+128602us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:18.272074 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:18.272136 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:19.216407 (+944271us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:19.408797 (+192390us) raft_consensus.cc:1685] finished
0709 13:05:19.408829 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.409107 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10138,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":11266,"spinlock_wait_cycles":12425600,"thread_start_us":94060,"threads_started":1,"wal-append.queue_time_us":241272}
I20260709 13:05:19.330324 13233 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.412153 13489 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.422565 13270 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:19.374428 13273 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.424789 13277 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.342859 13298 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.465368  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.465404 13476 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:19.466055 13529 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:05:19.466711  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.477001  9854 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:19.468731 13520 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.460011 13500 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.433593 13505 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.429971 13285 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.437840 13383 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.485580  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.464624 13489 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.494067  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.495041  9264 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:19.497133  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:19.499105  9121 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:19.484359 13528 raft_consensus.cc:697] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:19.504215 13331 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:19.504832 13233 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.506311 13273 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.508121  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.508785  9264 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:19.508988 13298 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.509150  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2604) took 1102 ms. Trace:
I20260709 13:05:19.509595 13383 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.481350 13334 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:05:19.511219 13489 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.490913 13505 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.513345  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:19.513986  9271 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:19.478677 13476 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.509552  9263 rpcz_store.cc:276] 0709 13:05:18.406629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.406977 (+   348us) service_pool.cc:224] Handling call
0709 13:05:18.407263 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.506928 (+1099665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.506961 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.507003 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:19.507036 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.508926 (+  1890us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000ec4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:19.508988 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.509094 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2528618752}
I20260709 13:05:19.475219 13277 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:19.515033  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:19.518586  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.490403 13500 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:19.523164  9828 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:19.524648  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:19.525803  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.529946  9828 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:19.533403  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.534309  9191 raft_consensus.cc:2410] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:05:19.534781 13285 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.535219  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.535373 13478 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:19.536998 13478 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:19.537415 13478 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:19.533455 13467 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:19.535933  9191 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:19.540783  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.542677 13277 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.542821  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.544135 13285 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.544665 13285 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:19.545027  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.504843 13331 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.547096  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.502236 13493 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.551896  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:19.549005  9191 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:19.554060  9195 raft_consensus.cc:2376] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6. Re-sending same reply.
I20260709 13:05:19.559478 13476 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.560716  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.562933  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.515654 13503 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.564260  9191 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:19.567253  9828 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:19.518009 13298 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.560546 13233 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:19.549048 13466 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.581034 13233 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:19.574422  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.582154  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2372) took 1287 ms. Trace:
I20260709 13:05:19.581969 13233 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.582379  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.583015  9263 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:19.582696  9120 rpcz_store.cc:276] 0709 13:05:18.294570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.294698 (+   128us) service_pool.cc:224] Handling call
0709 13:05:19.581843 (+1287145us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b540023c3d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:19.582048 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2017966848}
I20260709 13:05:19.572192 13467 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:19.576437  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.583945 13467 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.584147  9191 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:19.584345 13467 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:19.585114  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.586253  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.587288  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:19.564388 13514 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.343s	user 0.000s	sys 0.003s
I20260709 13:05:19.590572  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:19.596052  9191 raft_consensus.cc:2410] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:05:19.590788  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.592621  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.592705  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:19.593303  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:19.597286  9115 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:19.597821  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:19.598644  9114 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:19.600024  9263 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:05:19.549081 13331 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.600912  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:19.600870  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.601580  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.603113 13538 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:19.603446  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:19.614517  9856 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:19.503294 13270 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.565868 13277 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:19.609184  9162 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:19.609583  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.610509  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:19.615909  9826 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:19.616645  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.617624  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.619580  9854 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:19.613359 13526 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:19.550971 13500 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.612676 13451 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.626853  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.630524  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.631191  9192 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:19.632148  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.633193  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.634227  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.635200  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:19.625403 13413 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:19.636234 13413 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.635679  9192 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:19.638659  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:19.643004  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.643358  9828 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:19.643896  9272 raft_consensus.cc:2410] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:05:19.644851  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.646126  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.621372 13503 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:19.621881 13277 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:19.624099 13331 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:19.623939 13526 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.650002  9272 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:19.595121 13298 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.652770  9123 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:19.635380 13467 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.654909  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:19.637245 13500 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.625644 13349 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:19.650942 13331 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.663551  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:19.664633  9272 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:19.621102 13538 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.651332 13277 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:19.658970 13526 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:19.656404 13111 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:19.665323 13331 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:19.665797 13334 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:19.629536 13528 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.742027 13111 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:19.770625 13520 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.665447 13349 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.776953 13283 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.858160 13451 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:19.868908  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 240475us
W20260709 13:05:19.869436  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 253588us
I20260709 13:05:19.784071 13503 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:19.667275 13448 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.896135  9192 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:19.806906 13285 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.911346  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:19.906667  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 264078us
I20260709 13:05:19.739459 13274 consensus_queue.cc:1059] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.155s
I20260709 13:05:19.890158 13111 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:19.914367  9268 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:19.870028 13466 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:19.916658  9270 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:19.791044 13334 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:19.920765  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 199669us
I20260709 13:05:19.925994  9856 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 11 LEARNER]: Advancing to term 12
W20260709 13:05:19.920549 13377 consensus_meta.cc:287] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.001s
I20260709 13:05:19.917050 13413 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:19.921887 13180 consensus_meta.cc:287] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.544s	user 0.012s	sys 0.000s
I20260709 13:05:19.929816  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:19.918502 13546 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:19.940972  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:19.910665 13503 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.931344 13499 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:19.920742 13466 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:19.995401  9856 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 12 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:19.923324 13504 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:19.997993  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1236) took 1737 ms. Trace:
I20260709 13:05:19.996721  9271 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:05:19.936131 13524 consensus_meta.cc:287] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.525s	user 0.005s	sys 0.006s
W20260709 13:05:19.997340 13543 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:05:19.927798 13545 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:05:20.001744 13274 raft_consensus.cc:3055] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:20.097594 13545 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:20.097643 13274 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:20.098366 13545 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:20.098771 13545 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:20.094331 13521 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.099706 13545 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.100466 13521 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.101472 13545 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:20.088538 13298 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.994935 13546 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:20.121277 13546 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:20.125542 13469 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.125990 13349 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.126263 13469 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:20.089205 13504 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:20.128764 13469 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.106957 13503 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:19.931558 13233 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.160s
I20260709 13:05:20.130139 13504 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:19.997710 13467 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.107931 13524 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.131317 13467 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:20.131851 13467 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:20.132452 13448 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.133208  9263 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:20.132758 13467 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.136412 13283 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:20.136570  9263 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:20.138033 13334 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.138875 13283 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.139061 13538 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.139485 13283 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.098973 13274 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.141812 13283 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:20.109375 13478 consensus_meta.cc:287] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.572s	user 0.001s	sys 0.012s
I20260709 13:05:20.142772 13478 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:19.945573 13180 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:20.145489 13180 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.148516  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.067401 13499 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.152590 13499 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.155063  9191 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:20.096489 13520 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:20.117223 13371 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:20.157416 13371 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.158561 13478 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:20.158854  9119 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 2374) took 2106 ms (client timeout 1704 ms). Trace:
W20260709 13:05:20.159260  9119 rpcz_store.cc:269] 0709 13:05:18.052592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.052972 (+   380us) service_pool.cc:224] Handling call
0709 13:05:20.158651 (+2105679us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54001202d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:20.158756 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276350848}
I20260709 13:05:20.094947  9861 rpcz_store.cc:276] 0709 13:05:18.260952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.261174 (+   222us) service_pool.cc:224] Handling call
0709 13:05:18.261487 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.384085 (+122598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:18.384116 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:18.384177 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:18.384209 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:19.997750 (+1613541us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b540070f154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:19.997803 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:19.997935 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281471872}
I20260709 13:05:20.119248 13273 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.160431 13371 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.117646 13466 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.142125 13274 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:20.160696 13273 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.163602  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:20.165074  9861 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:20.132546 13524 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:20.168262  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 267445us
I20260709 13:05:20.173651  9825 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:20.176939  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 236041us
I20260709 13:05:20.171274 13156 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:20.180920  9826 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.191771  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:20.192713  9856 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:20.194566  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.184278  9825 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.204200  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.200628  9825 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:20.218729  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.220906  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:20.173352 13469 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.152621 13521 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.182303 13413 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:20.181463 13524 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:20.229939  9856 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:20.197034 13377 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.190024 13466 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.204783 13499 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:20.207875 13545 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:20.174401 13277 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.248693 13273 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.249153  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:20.212231 13504 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.206080 13547 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:20.232832 13558 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:20.258224  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.265537  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2185) took 1075 ms. Trace:
I20260709 13:05:20.250154 13298 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:20.277302  9855 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:20.277890  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1043) took 1591 ms. Trace:
I20260709 13:05:20.278445  9853 rpcz_store.cc:276] 0709 13:05:18.686362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.691518 (+  5156us) service_pool.cc:224] Handling call
0709 13:05:18.691847 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:18.939052 (+247205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.275107 (+1336055us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54006a0254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:20.277831 (+  2724us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":566289280}
I20260709 13:05:20.273824  9192 rpcz_store.cc:276] 0709 13:05:19.190267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.910051 (+719784us) service_pool.cc:224] Handling call
0709 13:05:19.910468 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.910731 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:19.993721 (+ 82990us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.265144 (+271423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:20.265211 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:20.265260 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.265339 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.265471 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":196981,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":8230,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":103,"thread_start_us":343843,"threads_started":2}
I20260709 13:05:20.259157 13545 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:20.281044  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:20.246475 13524 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.232988 13469 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.284400  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:20.285121  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:20.286873  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.256528 13377 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.263392 13499 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:20.292053  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.292260 13499 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.292799  9853 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:20.259112 13466 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.282815 13545 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:20.265677 13331 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:20.308429 13482 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 2.024s	user 0.001s	sys 0.014s
I20260709 13:05:20.310101 13331 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.281332 13558 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:20.311321  9194 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:20.314406  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.313877  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:20.312777  9089 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.317392  9116 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:20.305809 13111 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.329258  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.330152 13111 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.332026 13334 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.333050 13334 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.334811  9190 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:05:20.335985 13270 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:20.337051 13270 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:20.338357  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:20.339082  9192 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:20.349480 13524 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.352066 13273 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:05:20.357189  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:20.358915 13526 consensus_meta.cc:287] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.575s	user 0.004s	sys 0.006s
I20260709 13:05:20.359514 13526 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.361006  9272 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:20.363131 13156 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:20.363708 13156 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.365962 13156 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:20.368649  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 55952.3us
I20260709 13:05:20.370321  9089 leader_election.cc:400] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:05:20.370780  9089 leader_election.cc:403] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:20.371513 13180 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:20.372517 13180 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:05:20.372485 13546 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.373715  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.374701  9855 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:20.379144 13362 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.329454  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:20.321439  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.326562 13469 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.388969  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:20.389667  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:20.386241  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 54813.5us
I20260709 13:05:20.387547  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.271311 13504 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.392318  9119 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:20.391214  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.394438  9861 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:20.398286  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:20.399654  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.400945  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:20.401767  9853 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:20.403379  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.362433 13526 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.411280  9861 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:20.409391  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:20.375488 13546 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.415048  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:20.426025  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.449092  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.459617  9164 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:20.461864  9116 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:20.464677  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:20.410303 13524 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:20.422276 13493 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:20.468688  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.420521 13362 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:20.475638  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:20.476389  9122 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:20.478833  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.426875 13371 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:20.469422 13334 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.472149 13469 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:20.478394 13156 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:20.488749 13371 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:20.506781  9198 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:05:20.476015 13362 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:20.513408  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 286858us
I20260709 13:05:20.490797 13493 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:20.410564 13558 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.522941  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:20.525007  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:20.527083  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:20.527539  9116 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:20.525142  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:20.483979 13201 consensus_queue.cc:1059] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:05:20.541464  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:20.542528  9271 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:20.513482 13371 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:20.515905 13362 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:20.514218 13471 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.565451  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:20.562453  9122 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:20.513969 13277 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.567304 13478 raft_consensus.cc:3055] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:20.580719  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 783) took 1612 ms. Trace:
I20260709 13:05:20.566175 13546 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:20.581177  9196 rpcz_store.cc:276] 0709 13:05:18.968209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:18.968449 (+   240us) service_pool.cc:224] Handling call
0709 13:05:20.580515 (+1612066us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400399c54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:20.580657 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391452032}
I20260709 13:05:20.588594  9271 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:20.591931 13471 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.592919  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.653628  9122 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:20.594398  9116 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:20.595009  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:20.595184  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.655596  9120 raft_consensus.cc:2393] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:20.596835  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.597826  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.657153  9119 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:05:20.601748  9190 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2182) took 1412 ms (client timeout 1602 ms). Trace:
I20260709 13:05:20.603369  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:20.658082  9190 rpcz_store.cc:269] 0709 13:05:19.189455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.273671 (+ 84216us) service_pool.cc:224] Handling call
0709 13:05:20.601468 (+1327797us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400398d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:20.601638 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2413739776}
I20260709 13:05:20.606832 13467 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:20.611375  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:20.613621  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.614946  9115 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:20.616046  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:20.620904  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.621165 13524 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:20.623555  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.626683  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2610) took 1138 ms. Trace:
I20260709 13:05:20.637516  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.573518 13377 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.595824  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.656464  9121 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:20.668164  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 911) took 3363 ms. Trace:
I20260709 13:05:20.672079  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:20.674273  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.679150  9117 raft_consensus.cc:2410] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:20.676100  9859 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:20.642455  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.679189  9269 rpcz_store.cc:276] 0709 13:05:17.270162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.270298 (+   136us) service_pool.cc:224] Handling call
0709 13:05:18.372772 (+1102474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:19.300280 (+927508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.305743 (+1005463us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b54000eb0d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:20.633336 (+327593us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000eb0d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:20.633555 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4542564608}
I20260709 13:05:20.680091  9114 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
W20260709 13:05:20.680398  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 69155.8us
I20260709 13:05:20.682060  9087 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.672559  9272 rpcz_store.cc:276] 0709 13:05:19.487859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.688106 (+200247us) service_pool.cc:224] Handling call
0709 13:05:19.688554 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:20.361449 (+672895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.626374 (+264925us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54000fafd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:20.626608 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546013056}
I20260709 13:05:20.694844  9196 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:20.695837  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.696705  9857 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:20.577030 13478 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:20.699946  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.701794  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.706215  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:20.706697  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.708858  9857 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:05:20.709380  9857 pending_rounds.cc:85] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 9
I20260709 13:05:20.709681  9857 pending_rounds.cc:107] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:05:20.710007  9857 raft_consensus.cc:2889] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:20.712020  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.712692  9117 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:20.713536  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:20.674961 13538 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:20.691316 13270 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:20.675468 13555 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:20.694404 13180 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:20.719724 13521 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:20.700388 13466 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.671829 13471 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:20.715482 13561 log.cc:927] Time spent T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 13:05:20.705420 13273 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:20.718695 13504 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.667483 13493 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.746856  9190 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:20.729467  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 751) took 1527 ms. Trace:
I20260709 13:05:20.672946 13546 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.728341  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1207) took 1336 ms. Trace:
I20260709 13:05:20.691092 13524 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:20.721206 13564 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:05:20.718106 13270 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:20.703521 13372 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:20.750135  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:20.748589  9123 rpcz_store.cc:276] 0709 13:05:19.202307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.216998 (+ 14691us) service_pool.cc:224] Handling call
0709 13:05:19.217298 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.661783 (+444485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.729247 (+1067464us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540023cdd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:20.729409 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992731520}
I20260709 13:05:20.719323 13538 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:20.764370  9089 leader_election.cc:336] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.602s (SENT)
I20260709 13:05:20.754225  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:20.726227 13555 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.773857  9854 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:05:20.748299 13273 raft_consensus.cc:697] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:20.752431  9860 rpcz_store.cc:276] 0709 13:05:19.391641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:19.391859 (+   218us) service_pool.cc:224] Handling call
0709 13:05:19.392177 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:19.473393 (+ 81216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:19.473437 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:19.473494 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:19.473538 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.728091 (+1254553us) spinlock_profiling.cc:243] Waited 81 ms on lock 0x7b54005ee554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:20.728146 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.728282 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186210176}
I20260709 13:05:20.803416  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:20.804021  9190 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:20.764310 13270 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:20.811580  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:20.812758  9193 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:20.829412 13471 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.833423 13546 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.835810  9193 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:20.838804  9190 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:20.841346 13555 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:20.843900  9193 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:05:20.858312 13568 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:05:20.765998 13372 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:20.878472  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 181810us
W20260709 13:05:20.880244  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 206047us
I20260709 13:05:20.879525  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.885277  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:20.808437 13273 consensus_queue.cc:237] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:20.886173  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 98623us
W20260709 13:05:20.889969  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:20.888404 13361 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:20.897109  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 792) took 1620 ms. Trace:
I20260709 13:05:20.898542  9195 rpcz_store.cc:276] 0709 13:05:19.276859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.161407 (+884548us) service_pool.cc:224] Handling call
0709 13:05:20.161706 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.162063 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:20.162354 (+   291us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.266117 (+103763us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:20.269650 (+  3533us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.269860 (+   210us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:20.270320 (+   460us) log.cc:844] Serialized 14 byte log entry
0709 13:05:20.270677 (+   357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.270798 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.371011 (+100213us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b48000030d8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:05:20.590650 (+219639us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b18003acd80. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:20.896893 (+306243us) raft_consensus.cc:1685] finished
0709 13:05:20.896918 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.897050 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11716,"prepare.queue_time_us":4953,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":3522,"raft.queue_time_us":1206,"raft.run_cpu_time_us":2715,"raft.run_wall_time_us":103179,"spinlock_wait_cycles":12767872,"thread_start_us":102785,"threads_started":1,"wal-append.queue_time_us":138952}
W20260709 13:05:20.903802 13569 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:05:20.886809 13526 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:05:20.888557 13521 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:05:20.916718  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.919486  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2146) took 1597 ms. Trace:
I20260709 13:05:20.920387 13524 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:20.921399 13577 consensus_queue.cc:1059] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:20.921993 13372 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:20.922693 13570 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:05:20.924925 13505 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.925037  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:20.925567 13545 consensus_meta.cc:287] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.000s
I20260709 13:05:20.925500  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:20.925923  9117 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:20.926301  9123 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:20.926203 13545 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.897346 13579 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:20.928274 13579 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.930159 13578 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:05:20.931113 13526 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:20.931610 13506 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.919997  9189 rpcz_store.cc:276] 0709 13:05:19.322102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.417931 (+1095829us) service_pool.cc:224] Handling call
0709 13:05:20.424543 (+  6612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.424835 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:20.424861 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:20.425394 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.678943 (+253549us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:20.678983 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.679066 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.919255 (+240189us) raft_consensus.cc:1685] finished
0709 13:05:20.919280 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.919422 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9476,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":9826,"thread_start_us":195316,"threads_started":1,"wal-append.queue_time_us":353252}
I20260709 13:05:20.938506  9161 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:20.905411 13180 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:20.910941 13361 raft_consensus.cc:697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:20.945881  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:20.948630  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 137402us
I20260709 13:05:20.952409  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:20.952596  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:20.957855 13361 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.960477 13579 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.945255 13180 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:20.961344 13180 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:20.965381 13156 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:20.966056 13521 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:20.966867  9197 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:05:20.966595 13521 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.967993 13362 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.968667 13521 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.969163 13521 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:20.969501 13257 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:20.969587 13521 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.970245 13257 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:20.947180 13471 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:20.960270  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2186) took 1641 ms. Trace:
I20260709 13:05:20.970374 13362 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.941053 13371 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.980522  9267 raft_consensus.cc:2368] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:05:20.981957 13270 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:20.974829  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:20.974466  9192 rpcz_store.cc:276] 0709 13:05:19.318964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.339837 (+1020873us) service_pool.cc:224] Handling call
0709 13:05:20.340255 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.342162 (+  1907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:20.342217 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:20.342838 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.546455 (+203617us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:20.546520 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.546625 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:20.960065 (+413440us) raft_consensus.cc:1685] finished
0709 13:05:20.960085 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:20.960214 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1235,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":103,"thread_start_us":202988,"threads_started":1,"wal-append.queue_time_us":382642}
W20260709 13:05:20.985536  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 97739us
I20260709 13:05:20.973117  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2150) took 1510 ms. Trace:
I20260709 13:05:20.985786  9161 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:20.996310  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:20.998569  9825 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:20.986099  9197 rpcz_store.cc:276] 0709 13:05:19.462509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.487894 (+1025385us) service_pool.cc:224] Handling call
0709 13:05:20.493193 (+  5299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.967270 (+474077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.972837 (+  5567us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b5400217d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:20.973056 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1088785792}
I20260709 13:05:21.000054  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:21.000962  9861 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 6: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:21.000000  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:21.005322  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:21.007130  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:21.008235  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:21.020745  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:21.022383  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:20.971637 13521 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:20.971930 13518 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:21.004146 13377 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:21.031190  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:21.006073 13331 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:21.035423  9853 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:05:21.011090 13257 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:21.021088 13362 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:21.027534 13476 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:21.056260  9825 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:20.974965 13156 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:05:21.042438  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:21.045260  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.052592  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.026111 13524 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:21.046890  9858 raft_consensus.cc:1217] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 12 LEARNER]: Deduplicated request from leader. Original: 8.5->[8.6-12.8]   Dedup: 8.6->[10.7-12.8]
I20260709 13:05:21.048184  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:21.025106 13555 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:21.021745 13471 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:21.057559  9859 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:21.050952  9826 leader_election.cc:400] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:05:21.059144  9826 leader_election.cc:403] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:05:21.044101 13577 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:21.059659 13577 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.031307 13546 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:21.062559  9853 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:21.067791  9117 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:05:21.058040  9858 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 6
I20260709 13:05:21.068588  9858 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:05:21.068738  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:21.069428  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.065184  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.064066  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.068116  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:21.067435  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:21.070911  9121 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:21.068856  9858 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 12 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:21.065465  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.070912  9195 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:21.067544  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.073288  9857 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
W20260709 13:05:21.074251  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 189655us
I20260709 13:05:21.074826  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:21.076382  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.076924  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:21.077521  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.078524  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.079490  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.033561 13377 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.078333  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:21.082706  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 155942us
I20260709 13:05:21.036242 13331 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:21.115065  9123 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:21.117542  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:21.063445 13476 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.062493 13471 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.122342  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:21.129325  9272 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:21.117590  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:21.063277 13524 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:21.116456  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.119328  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.165511  9857 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:21.144375  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:21.145833  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:21.148355 13504 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:21.149953 13577 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.150695 13232 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:21.151546 13270 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.165431  9263 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:21.039304 13521 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 13:05:21.079017 13274 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.083660 13581 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:21.115036 13566 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:05:21.115727 13377 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.118854 13111 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:21.117590 13331 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.156987 13180 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.158464  9859 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:21.036823 13518 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.126561 13545 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:21.162117  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.183666 13524 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.134320 13476 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:21.154071 13565 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.571s	user 0.002s	sys 0.000s
I20260709 13:05:21.191414  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:21.279102  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 228751us
I20260709 13:05:21.192473  9263 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:21.197242  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.281389  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.279587  9271 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:21.283052  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:21.197373  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 74686.4us
I20260709 13:05:21.143366  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:21.285181  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 141957us
I20260709 13:05:21.197875  9862 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:21.133821 13471 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.190248 13583 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:21.185464 13581 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:21.225251  9265 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:21.193749 13585 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:05:21.238335 13518 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.246210  9272 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:21.185369 13361 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:05:21.256448 13257 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:21.259588 13377 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.261270  9193 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:21.268028  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.297538  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:21.270586 13331 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.272667  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:21.303381  9190 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.303645  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:21.278923 13526 raft_consensus.cc:3055] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:21.301664  9089 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:21.309036  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:21.313990  9193 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
W20260709 13:05:21.313869  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 257643us
I20260709 13:05:21.282300 13577 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:05:21.295970 13582 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.294648 13270 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:05:21.293380  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 166161us
I20260709 13:05:21.296448 13581 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.333135  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.292683 13362 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:21.334234  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.334723  9825 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:21.334892  9271 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:21.310125 13171 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:21.335076  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:21.337446  9196 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:05:21.338579  9265 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:21.311810  9235 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:21.341203  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.345494  9263 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:21.319500 13505 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:21.313578 13526 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:21.300451 13466 consensus_queue.cc:1059] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:05:21.321753 13180 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:21.352720  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:21.356514  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 44871us
I20260709 13:05:21.342809 13578 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:21.358861  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.298661 13583 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.391913 13538 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:21.393898 13583 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.397030 13371 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.353629 13505 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.400712 13505 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.401497 13582 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:21.352664 13545 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:21.363168 13595 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:21.363718  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2144) took 2080 ms. Trace:
I20260709 13:05:21.368342 13466 raft_consensus.cc:3055] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:21.368882 13597 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:21.369429 13274 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:05:21.369725  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:21.411404 13597 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:21.411928 13597 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.412743  9197 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:21.413354 13581 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.413851  9197 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:21.370231 13578 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.415942 13466 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:21.416708 13578 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:21.371943 13331 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:21.372072 13521 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:21.417027 13466 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.418154 13521 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:21.418465 13466 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.418540 13521 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:21.407598 13595 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:21.419533 13521 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.308262 13257 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.421097 13595 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.422927 13595 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:21.381400  9190 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:21.384333 13362 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.322527 13566 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:05:21.346134 13171 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.389456 13526 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:21.405998  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48986.3us
I20260709 13:05:21.475606  9828 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:21.407922  9191 rpcz_store.cc:276] 0709 13:05:19.283817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.163631 (+879814us) service_pool.cc:224] Handling call
0709 13:05:20.165856 (+  2225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:20.166201 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:20.166239 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:20.172040 (+  5801us) log.cc:844] Serialized 26 byte log entry
0709 13:05:20.274958 (+102918us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:20.275000 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:20.275073 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:21.153460 (+878387us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:21.354655 (+201195us) raft_consensus.cc:1685] finished
0709 13:05:21.354683 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:21.363647 (+  8964us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1496,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":5173,"spinlock_wait_cycles":45217536,"thread_start_us":66374,"threads_started":1,"wal-append.queue_time_us":339481}
I20260709 13:05:21.430187  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.433940  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:21.436316  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:21.438681  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.481516  9115 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.421419 13521 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:21.451699  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 46129.3us
I20260709 13:05:21.425343  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:21.483922  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 58660us
I20260709 13:05:21.484658  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:21.484736  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:21.460500  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:21.485545  9857 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.462442  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.472793  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:21.486930  9117 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.415688 13581 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:21.487219  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.490077  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1247) took 1228 ms. Trace:
I20260709 13:05:21.490469  9856 rpcz_store.cc:276] 0709 13:05:20.252435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.273533 (+ 21098us) service_pool.cc:224] Handling call
0709 13:05:21.480908 (+1207375us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54002242d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:21.481029 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765794560}
I20260709 13:05:21.491139  9828 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:21.425107 13111 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:05:21.450448 13545 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:21.494681  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:21.503367  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:21.492236  9089 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:21.504606  9828 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:21.501137  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.506753  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:21.507309  9265 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.474529 13171 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:21.496102  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.507891  9828 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:21.517457 13524 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.494158 13592 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:05:21.519860  9265 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:21.517470  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:21.524187  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.524735  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.492283 13526 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:21.511596 13336 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:21.489823 13506 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:21.546665  9115 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:21.502785 13413 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:21.494887 13582 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:21.565248  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:21.552795  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45048.3us
I20260709 13:05:21.564638  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:21.489529 13558 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:05:21.572340  9116 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:21.573482  9263 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:05:21.573829  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.525789 13467 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:21.524362 13277 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:21.577797  9265 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:21.581583  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.581975  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.519728 13257 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.502338 13273 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:05:21.591845  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:21.564121 13371 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:21.565877 13233 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:21.572860 13582 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:21.605564  9861 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:21.609807  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 100405us
I20260709 13:05:21.525568 13476 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.580616 13336 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:21.619297  9826 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:21.578166 13581 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.582448 13597 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.641325  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.634989  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.582304 13596 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:21.642720  9123 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:21.642822 13596 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:21.650271  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:21.606251 13371 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:21.657683  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.658420  9115 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:21.617975 13582 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:21.660416  9189 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:21.639652 13277 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:21.662761  9122 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:21.663898  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:21.918111  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 254295us
I20260709 13:05:21.698087  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:21.805404  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:21.739866  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 171579us
I20260709 13:05:21.694052 13277 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:21.644963 13538 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:21.856371  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.637179 13601 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:21.801570 13606 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:05:21.809222 13596 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.719092 13609 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:21.696467 13233 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:21.767827  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:21.777072 13605 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:05:21.793066 13597 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:21.662809 13371 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:21.913324  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:21.913889  9235 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:21.788720 13476 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.794181 13471 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.648002 13581 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.654717 13545 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.869535  9857 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
W20260709 13:05:21.984895  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 60991.2us
I20260709 13:05:21.773748 13336 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.850426  9087 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:21.865415  9117 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:21.824726 13505 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.785519 13524 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.914426  9826 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:21.897691  9267 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:21.697000 13608 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:21.894464 13592 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:21.914341 13546 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:21.916605 13349 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:21.917764 13602 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:21.916049 13171 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.924288 13617 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:21.985471 13462 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:21.970817 13471 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.972752  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:21.977593  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:21.971423  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 57642.7us
W20260709 13:05:22.053720  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 130204us
I20260709 13:05:22.054910  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:21.970924 13581 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:21.926102 13277 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:21.969379 13597 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:21.934836 13538 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.969954  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.969900  9115 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:21.982118 13371 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:21.983878 13566 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:22.091650 13566 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:21.984031 13521 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.092753 13521 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:22.094890 13521 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:21.969851 13612 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:22.096263 13612 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:22.097314 13612 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:22.098726 13608 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:21.991885  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 80272us
I20260709 13:05:22.101364  9115 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:22.102574 13371 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.103219  9272 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:22.103284  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.991788  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:21.993784  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 143450us
I20260709 13:05:22.108978  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:22.113852  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:21.914974 13582 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:22.001300  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.937057 13233 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:21.926581 13615 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:22.122083 13602 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:22.053061  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 60172.3us
I20260709 13:05:22.122242 13615 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.056200  9235 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:22.123723 13582 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:22.124400  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 68297.9us
I20260709 13:05:22.124785 13615 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.125209  9235 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:22.059175 13362 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:22.126279 13362 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:22.057289  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:22.121097  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.128790  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:22.002051 13466 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:22.060377  9857 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:22.061194  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.131264  9121 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:22.003432 13592 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.003822 13349 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.003620 13546 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.057562 13617 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.065841 13616 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:22.139842 13623 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:22.140050 13616 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:22.141841 13546 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.142074 13616 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.076913 13505 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.078513 13171 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.084160  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:21.929446 13601 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:21.997516  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.148401  9120 raft_consensus.cc:2368] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10.
I20260709 13:05:22.149547  9120 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:22.150867 13592 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:21.939960 13609 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.115381 13581 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:22.120958  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:22.120973  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.121397  9119 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:22.005295 13613 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:05:22.157284  9115 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 6: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:22.137157 13349 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:22.126852 13233 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:22.156276  9117 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:22.129601  9853 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:22.129329 13362 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:22.133353 13521 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:22.172816 13362 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:22.139576 13617 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.180796  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:22.181634  9856 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.165220 13466 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:22.185256 13546 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:22.185551 13538 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:22.167150  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:22.188019  9853 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:22.188915  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:22.185653 13546 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:22.192919 13546 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.194885 13546 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.195989 13521 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.196494 13521 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:22.168167  9235 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:22.203030 13609 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.203612 13538 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:22.204896 13621 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:22.181064 13622 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:22.212194 13622 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:22.217581  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 49524.4us
I20260709 13:05:22.155453 13608 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:05:22.222915 13362 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.219645  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:22.225030 13538 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:22.226963  9235 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:22.227128 13506 raft_consensus.cc:3055] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:22.235813 13506 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:22.236371 13538 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:22.236881 13506 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:22.237888 13622 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.238150 13506 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.238444  9235 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:22.229002  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1080) took 1181 ms. Trace:
I20260709 13:05:22.230679 13478 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.233511 13233 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.234131  9122 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:22.234441 13277 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.241240  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:22.239919  9859 rpcz_store.cc:276] 0709 13:05:21.047355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.069183 (+ 21828us) service_pool.cc:224] Handling call
0709 13:05:22.228841 (+1159658us) spinlock_profiling.cc:243] Waited 88.4 ms on lock 0x7b54005ee554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:22.228944 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":203324288}
I20260709 13:05:22.234436  9856 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:22.247264  9235 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:22.266399  9089 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:22.267429 13466 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:22.268088 13466 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:22.269201 13466 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.270773  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.271445  9118 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:22.272397  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:22.276944  9854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1256) took 1792 ms (client timeout 1724 ms). Trace:
W20260709 13:05:22.282248  9854 rpcz_store.cc:269] 0709 13:05:20.470416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.620659 (+150243us) service_pool.cc:224] Handling call
0709 13:05:22.262280 (+1641621us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400963cd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:22.262437 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235361536}
I20260709 13:05:22.283674  9118 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:22.266652  9235 leader_election.cc:400] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10.
I20260709 13:05:22.288661  9235 leader_election.cc:403] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:22.291025  9235 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:22.260140 13538 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:22.293856  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:22.294963  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.299497  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:22.311369  9856 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:22.312940  9854 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:22.359658  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 60238.9us
W20260709 13:05:22.359954  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.287667 13371 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:22.287402 13478 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:22.372165  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.372432  9856 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:22.293488 13597 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.384475  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:22.395995  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.397305  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.366464 13524 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:22.369951 13595 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:22.297775 13596 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:05:22.377161 13371 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:22.297339 13528 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:05:22.417917 13362 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.424182 13596 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:22.424431 13233 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.424968 13528 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:22.421603 13371 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.436694 13528 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:22.448782 13528 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.449966 13528 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:22.420491 13524 raft_consensus.cc:697] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:22.452584 13524 consensus_queue.cc:237] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:22.433667 13596 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:22.460739 13596 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:22.462208 13596 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.428782  9856 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:22.402357 13595 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:22.504390  9118 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:22.513688  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1217) took 1458 ms. Trace:
I20260709 13:05:22.514219  9860 rpcz_store.cc:276] 0709 13:05:21.055570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.127734 (+ 72164us) service_pool.cc:224] Handling call
0709 13:05:21.154405 (+ 26671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.387411 (+233006us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:21.387722 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.577599 (+189877us) log.cc:844] Serialized 14 byte log entry
0709 13:05:21.714982 (+137383us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:21.715037 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:21.715084 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.513459 (+798375us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b540009ae54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:22.513507 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.513628 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46075,"prepare.queue_time_us":2356,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":52146,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4379,"spinlock_wait_cycles":570148224,"thread_start_us":302515,"threads_started":2,"wal-append.queue_time_us":220448}
W20260709 13:05:22.543066  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.468072 13595 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:22.576490  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1223) took 1068 ms. Trace:
I20260709 13:05:22.576937  9854 rpcz_store.cc:276] 0709 13:05:21.508350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.283454 (+775104us) service_pool.cc:224] Handling call
0709 13:05:22.283842 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.313541 (+ 29699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.576250 (+262709us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b5400018bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:22.576420 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66334976}
I20260709 13:05:22.578949  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:22.581790 13111 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:22.582392 13111 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.583714 13467 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:22.584617  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1085) took 1152 ms. Trace:
I20260709 13:05:22.584973  9861 rpcz_store.cc:276] 0709 13:05:21.432338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.513681 (+ 81343us) service_pool.cc:224] Handling call
0709 13:05:21.514086 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.609548 (+ 95462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.584334 (+974786us) spinlock_profiling.cc:243] Waited 91.3 ms on lock 0x7b54006c02d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:22.584560 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209895808}
I20260709 13:05:22.585744 13546 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:22.585858  9853 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:22.586266 13546 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.584345 13467 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.590296  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:22.593586 13597 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10.
I20260709 13:05:22.629500  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2381) took 1072 ms. Trace:
I20260709 13:05:22.630072  9116 rpcz_store.cc:276] 0709 13:05:21.556820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.556979 (+   159us) service_pool.cc:224] Handling call
0709 13:05:21.565453 (+  8474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.583439 (+ 17986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.629284 (+1045845us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b540034e750. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:22.629426 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18558208}
W20260709 13:05:22.640532  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:22.641047  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.642710 13362 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:22.651293 13362 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.653519 13362 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.647574 13377 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.661270 13274 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:22.661754 13274 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:22.662679 13274 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:22.664630  9862 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1239) took 2447 ms (client timeout 2005 ms). Trace:
W20260709 13:05:22.665102  9862 rpcz_store.cc:269] 0709 13:05:20.217053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.220580 (+  3527us) service_pool.cc:224] Handling call
0709 13:05:22.664387 (+2443807us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b5400716454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:22.664525 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1875464448}
W20260709 13:05:22.667141  9189 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2196) took 1611 ms (client timeout 1546 ms). Trace:
I20260709 13:05:22.668687  9855 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:05:22.655238 13521 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.670504  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1255) took 2220 ms. Trace:
I20260709 13:05:22.670859  9855 rpcz_store.cc:276] 0709 13:05:20.450473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:20.469655 (+ 19182us) service_pool.cc:224] Handling call
0709 13:05:22.668324 (+2198669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.669092 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.670283 (+  1191us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b5400716450. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:22.670420 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5018821504}
I20260709 13:05:22.672950  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2044) took 1157 ms. Trace:
I20260709 13:05:22.673336  9265 rpcz_store.cc:276] 0709 13:05:21.515524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.550863 (+ 35339us) service_pool.cc:224] Handling call
0709 13:05:21.551150 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.584616 (+ 33466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.672656 (+1088040us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b54000f9bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:22.672896 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60884736}
I20260709 13:05:22.650487 13371 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:22.667580  9189 rpcz_store.cc:269] 0709 13:05:21.060857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.122004 (+ 61147us) service_pool.cc:224] Handling call
0709 13:05:22.666932 (+1544928us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400423bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:22.667074 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135116032}
W20260709 13:05:22.677416  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.678464 13467 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.643193 13546 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.680143 13274 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:22.681535 13566 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.683401 13336 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:22.683986 13336 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.688191  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:22.688867  9116 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.690428  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1260) took 1204 ms. Trace:
I20260709 13:05:22.690966  9857 rpcz_store.cc:276] 0709 13:05:21.488379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.989925 (+501546us) service_pool.cc:224] Handling call
0709 13:05:21.990226 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.130875 (+140649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.690063 (+559188us) spinlock_profiling.cc:243] Waited 70 ms on lock 0x7b54007102d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:22.690345 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160905088}
W20260709 13:05:22.700920 13639 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:22.702064 13526 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:22.702687 13526 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.703339 13524 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:22.704130 13524 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:22.702632  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:22.704855  9121 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:22.705595 13524 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:22.706571 13524 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:05:22.708734  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.707541 13524 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:22.717073  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:22.718796 13597 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.719338 13597 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.719915  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.723484  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.727774 13521 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.728922 13601 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:22.732224 13526 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.733287 13524 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:22.739444 13610 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.959s	user 0.000s	sys 0.009s
I20260709 13:05:22.740003  9116 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:22.742703 13601 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.744755  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.745975 13371 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:22.746551 13171 consensus_queue.cc:1059] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:22.777025 13601 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.783396 13171 raft_consensus.cc:3055] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:22.783874 13171 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:22.748812  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2378) took 1200 ms. Trace:
I20260709 13:05:22.751176 13597 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.759307  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2197) took 1428 ms. Trace:
I20260709 13:05:22.777571  9235 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:22.777590  9122 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:22.784868  9114 rpcz_store.cc:276] 0709 13:05:21.548151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.548296 (+   145us) service_pool.cc:224] Handling call
0709 13:05:21.548607 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.548885 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:21.580070 (+ 31185us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.739146 (+1159076us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b2400168c68. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765347d9c4 00007f765347bacb 00007f7653636dc9 00007f7653636a55 00007f765363609d 00007f765626810e 00007f765627f88c 00007f765627ebb6 00007f765627e7d8 00007f7656288221
0709 13:05:22.748026 (+  8880us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b18003b5c60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f765628c4bc 00007f76535da48b 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a
0709 13:05:22.748366 (+   340us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:22.748432 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:22.748488 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.748578 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.748744 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31463,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2637,"raft.queue_time_us":4684,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":75262464,"thread_start_us":35649,"threads_started":1,"wal-append.queue_time_us":178370}
I20260709 13:05:22.785166  9197 rpcz_store.cc:276] 0709 13:05:21.330562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.414607 (+ 84045us) service_pool.cc:224] Handling call
0709 13:05:21.414831 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:21.415002 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:21.450214 (+ 35212us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.758636 (+1308422us) spinlock_profiling.cc:243] Waited 4.06 ms on lock 0x7b1800172420. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f7653571c0a 00007f765ccb1219 00007f76535f9a40 00007f76535f44e4 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287 00007f76566ff208
0709 13:05:22.758816 (+   180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:22.758888 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:22.758954 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.759048 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.759219 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3385,"prepare.queue_time_us":44639,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":1315627,"spinlock_wait_cycles":9330432}
W20260709 13:05:22.796818  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 640002us
I20260709 13:05:22.780515 13596 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:22.784475 13336 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.805521  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2172) took 1344 ms. Trace:
I20260709 13:05:22.807446  9198 rpcz_store.cc:276] 0709 13:05:21.461177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.156604 (+695427us) service_pool.cc:224] Handling call
0709 13:05:22.156895 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:22.157084 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:22.292731 (+135647us) log.cc:844] Serialized 14 byte log entry
0709 13:05:22.805137 (+512406us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:22.805211 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:22.805275 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.805344 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.805461 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":176976,"prepare.queue_time_us":135640,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":460159,"threads_started":2,"wal-append.queue_time_us":441858}
I20260709 13:05:22.792516 13371 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:22.775920 13528 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:05:22.791834 13504 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:22.796753 13232 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:22.793084 13171 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:22.828372  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.819643 13581 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:22.853787 13111 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:22.871610  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.873874  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:22.838912  9161 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:22.875797  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:22.860383 13581 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.880342  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:22.861812 13371 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:22.892642  9197 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:22.893234  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.894025  9191 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:22.869386 13597 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:22.886731  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:22.896229  9198 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:22.904148  9116 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:05:22.904990  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:22.877775 13476 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:22.869084 13232 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.907227  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.907829 13595 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:22.908600 13232 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.909181 13232 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:22.909646 13232 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:22.910097 13595 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:22.910512 13232 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:22.911039  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:22.823798 13596 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.910128  9123 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:22.911767  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 166319us
I20260709 13:05:22.914091  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.914983  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.915232  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.918120  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:22.918956  9862 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:22.863214 13504 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.925042  9189 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:05:22.894877 13171 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.894170 13593 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.930308  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.897055 13331 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:22.933872  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:22.938082  9191 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:22.887684 13478 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:22.941097  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.941803  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:22.897500 13274 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:22.897228 13468 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.930800  9190 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:22.944370  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.945461  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.946396  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:22.946872  9192 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:22.949337  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:22.950089  9859 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:05:22.950280  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:22.951268  9862 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:22.939571  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 1.45625e+06us
I20260709 13:05:22.916569 13476 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:22.952157  9861 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:22.955699  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.952742  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:22.953028  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.953436  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:22.956506  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:22.935320 13156 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.957604  9114 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:22.956454  9196 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 6.
I20260709 13:05:22.960291  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:22.960649  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.960862  9862 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:22.962095  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.916307 13597 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.962790  9194 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:22.965178  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.965399  9860 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:22.964609  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:22.970359  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1075) took 1936 ms. Trace:
W20260709 13:05:22.974169  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:22.976022  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:22.970757  9858 rpcz_store.cc:276] 0709 13:05:21.033887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:21.034030 (+   143us) service_pool.cc:224] Handling call
0709 13:05:21.046703 (+ 12673us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:21.072639 (+ 25936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:21.072683 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:21.073353 (+   670us) log.cc:844] Serialized 52 byte log entry
0709 13:05:22.617037 (+1543684us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b18002e9ae0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:05:22.758646 (+141609us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:22.758689 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:22.758768 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:22.970057 (+211289us) raft_consensus.cc:1685] finished
0709 13:05:22.970075 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:22.970302 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39616,"raft.queue_time_us":755,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":40536,"spinlock_wait_cycles":13451648,"thread_start_us":141423,"threads_started":1,"wal-append.queue_time_us":1812909}
I20260709 13:05:22.974545  9121 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:22.977990  9192 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:22.976653  9854 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:22.980567  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:22.980110 13493 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:22.982230 13493 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:22.983101 13601 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.984534 13493 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:22.985229 13371 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:22.987233 13596 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.938253 13593 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.980016 13257 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:22.988382 13257 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.955031 13274 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.990213 13593 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.959547 13468 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.992638 13581 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.994655  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:22.995522  9857 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:22.998489  9191 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:23.001152  9191 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:23.003722  9862 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:22.934967 13336 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:05:23.009778  9190 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:05:22.951915 13478 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:22.962360 13648 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.022289  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.027556  9189 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:23.023190  9194 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: [8.6-12.8]
I20260709 13:05:22.947482 13331 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.026988 13597 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.030431  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.026228 13504 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:22.964942 13156 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:23.026810 13478 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.051630  9114 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:23.053280  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:23.054224  9119 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:23.031505 13648 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.035955 13476 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:23.020565 13493 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:23.056473  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:23.037837  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.044293 13277 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:23.059514  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:23.056246  9195 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:23.050314 13596 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:23.051272 13504 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:23.065858 13648 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.060956 13581 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:05:23.056061 13336 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:23.067031 13336 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.068238 13566 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.069712 13336 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.063019 13601 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:23.070767 13601 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:23.071638 13601 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:23.062413  9089 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.546s (SENT) [suppressed 17 similar messages]
I20260709 13:05:23.074355 13257 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:23.081207  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 50854.5us
I20260709 13:05:23.082212  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:23.083700  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:23.084980  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.055341 13478 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
W20260709 13:05:23.098354  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 156616us
W20260709 13:05:23.100672  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 133775us
I20260709 13:05:23.101502  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:23.095260 13538 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:23.105145  9196 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:05:23.086222 13596 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:23.099852  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.092799 13171 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:05:23.060259 13493 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:23.101541  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 162205us
W20260709 13:05:23.119310  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.108440  9162 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.095923 13362 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:23.105509 13597 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:23.105175 13538 raft_consensus.cc:697] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:05:23.152717  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.140527 13648 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:23.108263 13595 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:05:23.127647 13362 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:23.144508 13257 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:23.109598 13478 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:23.169842 13478 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:23.174043 13331 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.175067  9164 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:23.176581 13653 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:05:23.177763 13653 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:23.177981 13521 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:23.178166 13653 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.178426 13521 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:23.178577 13653 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:23.179140 13377 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:23.179828 13377 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:23.180755 13336 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:23.181084 13493 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.181206 13336 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:23.181885 13232 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:23.182158 13336 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.156067  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:23.141693 13597 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:23.201212  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43672.9us
W20260709 13:05:23.204942  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48971.8us
I20260709 13:05:23.195935 13597 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:23.145246 13664 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:23.160053 13362 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:23.183537  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 121140us
I20260709 13:05:23.200067 13170 raft_consensus.cc:3055] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:23.179324 13521 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:23.221120 13581 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:23.234160  9119 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:23.221726 13466 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:23.223462 13664 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:23.224977 13232 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:23.199769 13331 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:05:23.171103 13478 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.229897  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:23.144970  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 43549.9us
I20260709 13:05:23.205237 13156 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:23.285753  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:23.293752  9123 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:23.299265  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.210268 13336 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.210225 13277 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.209215 13471 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:23.201980 13659 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 13:05:23.301229  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.180970 13377 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:23.227461 13170 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:23.326007  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:23.326164  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:23.327270  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:23.341572  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 111758us
I20260709 13:05:23.285779 13233 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:23.343380  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:23.286069 13622 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:23.346220  9119 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
W20260709 13:05:23.346004  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 44846.7us
I20260709 13:05:23.316447 13528 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:23.352448  9857 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:23.360114  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.336652 13646 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.226222 13538 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.296258 13466 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:23.291935 13581 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.362162 13476 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:23.319483 13371 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.044s
W20260709 13:05:23.383929  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 177786us
I20260709 13:05:23.360980 13156 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:23.330603 13277 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.362776 13668 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:23.375062 13377 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.375295 13171 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:23.355679 13233 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.371034 13595 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:05:23.386631 13156 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:23.413405  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:23.414718  9116 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:23.422045  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:23.422452  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.424077  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:23.411504 13171 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.435413  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:23.437259  9116 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:23.437274 13171 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.393751 13528 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:23.382622 13646 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:23.461699  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.414594  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:23.369654 13622 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:23.525560  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 165542us
W20260709 13:05:23.530118  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 347324us
I20260709 13:05:23.531090  9164 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:23.540859 13658 log.cc:927] Time spent T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.441s	user 0.001s	sys 0.003s
I20260709 13:05:23.542781 13478 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:23.548513  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.550722 13468 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.553081 13521 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:23.527567  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:23.555198  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.555935  9263 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:23.556020  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.555615  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.556874  9265 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:23.556938  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:23.557214  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.559446  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.560127  9272 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:23.561203  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.562311  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.562734  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:23.562901  9272 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:23.563293  9265 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:23.516921 13579 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:23.514089 13649 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:23.570773  9270 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:23.568632  9269 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:23.569211  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:23.572227  9265 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:23.573853  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.576328  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.577167  9266 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:23.578171  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:23.578724  9272 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:23.580842  9263 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:23.524242 13372 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.593008 13372 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.577586 13674 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:23.389834 13170 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.586006  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.586022  9268 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:23.598433  9265 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:05:23.600337  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:23.592017  9272 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:23.593360  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.584828  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.604351  9266 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:23.604970  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:23.609215  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.617772  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:23.614749  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.612576  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.618835  9269 raft_consensus.cc:2410] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:05:23.619434  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:23.614872  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.570173 13579 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:23.612238  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.623243  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.586386 13655 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:23.621979  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:23.625065  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:23.623555  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.611884  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.625186  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.576860 13622 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:23.594169 13674 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:23.600571 13478 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:23.676146  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:23.674886  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:23.684686  9264 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:05:23.681656  9196 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:23.623847  9197 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
W20260709 13:05:23.686540  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 118399us
I20260709 13:05:23.676095  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.681147  9189 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:05:23.681803  9190 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:23.600943 13274 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:23.676057 13372 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:23.702613  9266 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:23.601730 13331 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:23.703181  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.638269  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:23.713882  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:23.714409 13298 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:23.714547 13670 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
W20260709 13:05:23.715556 13669 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
W20260709 13:05:23.743310  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 221397us
I20260709 13:05:23.728783 13277 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:23.728946 13233 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:23.679138 13660 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
W20260709 13:05:23.676779 13661 log.cc:927] Time spent T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:05:23.797344 13596 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:23.678443 13665 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:05:23.797951 13596 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.544135 13653 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.799901  9191 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:23.801654 13653 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.803917 13677 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:23.608476 13156 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:23.809062 13633 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:05:23.686115 13662 log.cc:927] Time spent T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
W20260709 13:05:23.737792  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 99653.1us
I20260709 13:05:23.737800  9269 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:23.608912 13462 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
W20260709 13:05:23.739881  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 370992us
I20260709 13:05:23.675411 13362 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.779062  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.856334  9189 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
W20260709 13:05:23.857580 13336 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.000s
I20260709 13:05:23.858076  9267 raft_consensus.cc:2393] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:23.684448 13674 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:23.406441 13664 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.866402  9266 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:23.868702  9269 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:23.712165 13622 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:23.742942 13298 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.821239  9162 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.723224 13666 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.745316 13478 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:23.779935 13466 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:23.822871 13462 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:05:23.828611 13566 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.760s	user 0.007s	sys 0.004s
I20260709 13:05:23.830139  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.779531 13581 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:23.828611 13257 consensus_meta.cc:287] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.000s
I20260709 13:05:23.839192  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.840569  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.974948  9271 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
W20260709 13:05:23.841173  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.976084  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.841981 13597 consensus_meta.cc:287] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.000s
I20260709 13:05:23.879290 13277 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.745437 13601 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:23.956672 13622 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:23.847594 13646 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.810657  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:23.852088 13471 raft_consensus.cc:3055] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:23.854239 13156 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:23.879192  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 278908us
I20260709 13:05:23.867573  9266 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:23.931108  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:23.942104  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.998925  9828 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:23.957261  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 79604.7us
W20260709 13:05:23.999729  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:23.999866 13689 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:24.000797 13689 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:24.001127 13689 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:23.958132  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 94306us
I20260709 13:05:24.002185 13689 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:23.988731 13686 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:05:23.887511 13678 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.006735  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.978801 13566 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:24.008317  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.010396  9828 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.003404 13689 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:23.968513  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 147385us
I20260709 13:05:24.020238  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.021606  9162 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:23.821866 13650 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:23.979200  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.022537  9855 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:24.023789  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:23.979689  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:23.980365  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:23.980986  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.025661  9862 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:23.982545  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:23.845074  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:23.990599 13622 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:23.984325  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:23.846740 13596 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.959199 13478 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:24.034724 13478 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:24.034821 13682 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:24.036402 13649 raft_consensus.cc:2764] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:05:24.037650 13679 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:24.038175 13679 raft_consensus.cc:697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:24.038986 13679 consensus_queue.cc:237] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:23.969220 13462 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:23.990393  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 303165us
I20260709 13:05:24.046171  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:23.991555 13471 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:23.995554  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:23.974547 13257 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:23.974871 13648 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:24.001051  9826 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.016986  9192 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:24.010872 13372 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:23.857640 13362 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:23.970140 13666 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.026737  9268 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:23.983534  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:23.963673  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.026546  9860 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:23.975448 13673 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:05:24.031574  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:24.032199  9265 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:24.032469  9828 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:24.032893  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:23.988327  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:24.080086  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 47684.5us
I20260709 13:05:24.040282  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.080489  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.041287 13650 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.041706 13493 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.019035 13566 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.084149 13493 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:24.085492 13566 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:24.086257 13566 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:24.086621 13493 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:23.967808  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:24.088340  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 120661us
I20260709 13:05:24.089046 13650 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.089820  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:23.975171 13685 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.090708 13685 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.091686 13666 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.092912 13685 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:24.093493  9264 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:23.989974 13601 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.005093 13678 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:23.984942 13597 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.067272  9856 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:24.028304 13274 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:05:24.075675 13659 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:24.105822  9271 raft_consensus.cc:2410] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:05:24.049902  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.048704  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.068727  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.107301  9189 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
W20260709 13:05:24.107350  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 83706.1us
I20260709 13:05:24.108460  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.110251  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.056515  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.111303  9190 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:24.069422  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:24.018764 13594 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.016625 13624 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:24.113102 13698 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:24.113245 13624 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:24.113651 13698 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:24.113920 13678 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:05:24.114023 13698 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.114138 13594 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.116408 13594 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.117215 13336 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:24.071260  9121 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:24.086625 13566 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:24.120370 13646 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.120873 13566 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:24.121047 13646 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.071844  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.017771  9197 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.122696 13579 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.123492  9194 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:24.110746 13659 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.127840 13659 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:24.129002 13597 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.129783  9263 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:24.130540  9272 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:24.103730 13691 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:24.132256  9267 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:24.132550 13691 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:24.053982 13377 consensus_meta.cc:287] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.000s
I20260709 13:05:24.081267  9196 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:23.990056 13277 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.136735  9861 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:24.134706 13691 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.081689  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.045374 13468 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.137854 13468 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.066048 13298 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.051335 13462 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.139106 13462 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:24.072747 13521 consensus_meta.cc:287] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.003s
W20260709 13:05:24.140197  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.140399  9265 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:05:24.096657  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:24.141640  9268 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:24.142220 13377 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:24.143576 13601 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.145186 13646 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.145399 13579 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.096836  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.060503  9270 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:24.104826 13685 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:24.104908  9087 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:24.094959  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.121542 13471 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.136693 13170 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.138195 13277 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.028653 13233 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:24.139689  9193 raft_consensus.cc:2410] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:05:24.164702  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:24.148845  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44040.2us
I20260709 13:05:24.168918  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.168973  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:24.112424  9122 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:24.148870  9263 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
W20260709 13:05:24.148996  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 54123.7us
I20260709 13:05:24.152707  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.171147  9263 pending_rounds.cc:85] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 4
I20260709 13:05:24.171512  9263 pending_rounds.cc:107] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:05:24.148168 13685 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:24.158033  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49069.4us
I20260709 13:05:24.160094  9197 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:24.161357  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.175462  9164 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.171866  9263 raft_consensus.cc:2889] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:24.176167  9191 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:24.176498  9266 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:24.179306  9122 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:24.182966  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.183418  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.184916  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.189733  9825 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:24.193148  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.196097  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:24.197278  9164 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.199473  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.201237  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.202965  9117 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:24.203774  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.204381  9123 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:24.205051  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:24.206403  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 125426us
I20260709 13:05:24.207991  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.209961  9861 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:24.210891  9270 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:05:24.258440  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.258805  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.214910  9164 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.238723  9195 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:24.219332  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.229463  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.216269  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:24.267863  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 53040.9us
I20260709 13:05:24.217790  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:24.250222  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:24.270627  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.172659 13471 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.192764 13685 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:24.264524 13566 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.192682 13504 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.274145  9272 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:24.172628 13601 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:24.195142 13524 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.273572  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:24.295295  9116 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:24.157691 13646 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:24.216532  9162 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.205307 13648 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:24.310395 13648 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:24.310638 13594 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:24.310871  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 94404.7us
I20260709 13:05:24.240048 13362 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:24.311415 13648 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.228910 13672 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:24.312669 13672 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.312844  9162 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:24.248802 13677 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:05:24.255884 13257 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:24.313947 13257 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:24.198069 13666 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:24.277222 13156 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:24.280714  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.314913 13156 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:24.313236 13648 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.315359 13156 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.319959 13593 consensus_queue.cc:1059] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:24.281700  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:24.282385  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:24.282588  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.284338  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.172796 13521 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:24.288636  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.205503 13528 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:24.289886  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.297297  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.240329 13274 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:05:24.247133 13674 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.293787  9828 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:24.293834  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:24.299575  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 298581us
I20260709 13:05:24.299940  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.299988  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:24.300933  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.207052 13526 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:24.212000 13673 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:24.311069 13594 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.192840 13233 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.311968 13362 raft_consensus.cc:697] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:24.313079 13672 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.211264 13298 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:24.206540 13597 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:05:24.279891 13538 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:24.315213  9266 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:24.280975 13664 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:24.229084 13679 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:24.358354  9860 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:24.382885  9120 raft_consensus.cc:2410] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:24.358736  9861 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:24.359951  9197 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:24.192844 13681 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 13:05:24.250048 13468 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.273890 13650 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:24.274307 13493 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.362748  9271 raft_consensus.cc:2393] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:24.361346  9268 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:24.363590  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.360743  9270 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:24.364070  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:24.417644  9272 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:24.366734  9856 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:24.366933  9122 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:24.363581  9859 raft_consensus.cc:2410] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:24.209381 13596 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.124s
I20260709 13:05:24.369174  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:05:24.370102  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 55410us
I20260709 13:05:24.335035 13692 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:05:24.371405  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.426419  9196 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:24.371939  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:24.427150  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 58063.4us
I20260709 13:05:24.427330  9857 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:24.208762 13478 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.244963 13649 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.120s
W20260709 13:05:24.377293  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 83609.2us
I20260709 13:05:24.377339  9265 raft_consensus.cc:2376] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9. Re-sending same reply.
I20260709 13:05:24.321349 13682 raft_consensus.cc:3055] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:24.378996 13668 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:24.379619 13699 log.cc:927] Time spent T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:05:24.380260  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.384624  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.363677 13594 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.389750  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.390460  9190 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:05:24.448808  9190 pending_rounds.cc:85] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 5
I20260709 13:05:24.449124  9190 pending_rounds.cc:107] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:05:24.449419  9190 raft_consensus.cc:2889] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:24.427927  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.429566  9238 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:24.431208  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 148723us
I20260709 13:05:24.361605 13524 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.451438  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:24.431895  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 138140us
I20260709 13:05:24.423853 13666 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.301518 13504 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.385468 13462 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:24.432749 13521 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:24.381162 13298 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:24.446826 13690 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:24.446905 13232 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:05:24.456209 13701 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.456684 13277 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.457110 13466 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:05:24.457333 13362 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.509152 13298 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.459022 13692 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:24.462071 13698 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.457121 13233 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.514042 13698 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.514129 13233 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:24.514647 13233 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:24.515034 13233 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.447119 13689 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.517521 13701 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.517858 13689 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.490556 13521 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:24.520495 13689 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.464859 13478 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.466293 13700 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.522063 13674 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.521842 13700 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.467844  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.431070 13650 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:24.523759 13650 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:24.524410 13700 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.467123 13622 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.524641 13277 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.490738 13274 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.468925 13579 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:24.470359  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:24.430282 13696 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:24.527045 13274 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.492466  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.528620  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:24.529315  9190 raft_consensus.cc:606] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:24.493158  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.530380  9195 raft_consensus.cc:2410] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:05:24.531841  9853 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:24.493803  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48108.4us
I20260709 13:05:24.425266 13659 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.494652  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.495561  9197 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:24.496325  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.497246  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.436192 13682 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:24.503355  9263 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:24.445452 13668 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.430419 13493 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.490520 13462 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:24.526335 13622 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.527491 13696 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:24.471138 13691 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:24.489526  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 60046us
I20260709 13:05:24.493965  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:24.436239 13678 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:05:24.538671  9269 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:24.543136  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 75371us
I20260709 13:05:24.488633 13666 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.504019 13371 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:24.471724 13655 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:05:24.554018 13277 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:24.543282  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 72999.3us
I20260709 13:05:24.545513 13668 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:24.558082  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:24.753334  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 195326us
I20260709 13:05:24.754316  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:24.625530  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 197663us
I20260709 13:05:24.626952  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.635887  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.637503  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.519690 13701 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.755479  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.755978  9854 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:24.756901  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:24.556326 13692 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:24.763667  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 136837us
I20260709 13:05:24.747720  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.764468  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:24.747825  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:24.550076 13462 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:24.758919  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.768306  9857 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:24.768539  9266 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:05:24.769328  9857 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:24.759888  9862 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:24.554854 13696 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:24.765548  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:24.765570  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:24.770632  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:24.790835  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.771680  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:24.551177 13674 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:24.763161 13466 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:24.813237  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47741.2us
I20260709 13:05:24.817174  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:24.818331  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.818863  9197 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:24.819600  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.820334  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:24.821821  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:24.822288  9195 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:24.825037  9828 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:24.782889  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.827173  9828 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:24.826669  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.783515 13581 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.783862 13689 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:05:24.784391 13695 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.229s	user 0.000s	sys 0.000s
I20260709 13:05:24.785216 13597 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:24.830684 13466 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:24.827488 13703 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.460s	user 0.001s	sys 0.000s
I20260709 13:05:24.797950  9854 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:24.730988 13377 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.841387 13377 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.828662 13579 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:24.842265 13579 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:24.842664 13579 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.829725  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.763758 13362 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:24.756938 13690 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.831126  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:24.756834 13371 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:05:24.834163  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:24.807180 13372 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:05:24.825731 13674 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:24.876015 13674 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:24.876547 13710 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 13:05:24.878463 13233 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.768879 13682 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.886840 13622 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:24.887331 13622 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:24.887401 13524 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:24.889114 13493 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.891040 13659 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.836910 13650 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:24.826171 13698 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:24.892167 13691 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:05:24.839401 13708 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:24.851795 13377 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.893357 13708 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:24.847026 13701 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:24.893793 13708 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.893939 13701 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.894347 13701 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:24.840379 13649 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.895237 13649 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.896384 13298 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:24.841782 13466 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:24.828778  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.763962 13692 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:24.763398 13566 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:24.667989 13594 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:24.763554 13677 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:24.789887 13686 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:24.851378  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.812933 13277 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:24.811671 13171 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:24.832445 13601 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:05:24.912860  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 84151.8us
W20260709 13:05:24.931268  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 80019.9us
I20260709 13:05:24.852507 13362 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.922492 13372 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:24.937896  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.858712 13664 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:24.937906  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.943626  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:24.896981 13298 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:24.927891 13695 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:05:24.926069 13692 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:24.928079 13691 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:24.959496  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 176659us
I20260709 13:05:24.975414  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:24.937011 13171 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.942646 13601 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:24.946213 13362 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.947937 13566 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:24.944344 13493 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:24.957461 13298 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.972090 13524 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:24.892204 13650 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.957643 13713 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:24.982106 13171 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:24.999053 13601 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:25.008124 13466 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:25.022145 13170 consensus_meta.cc:287] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.859s	user 0.012s	sys 0.000s
I20260709 13:05:25.009055 13646 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.898109 13690 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:25.027850 13581 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:25.028154 13648 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.713s	user 0.012s	sys 0.001s
I20260709 13:05:24.939198 13712 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.151s
I20260709 13:05:24.979261 13664 consensus_queue.cc:1059] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:05:24.942945 13677 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:24.978790 13686 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:05:25.015839 13233 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:25.038022  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 65921.1us
I20260709 13:05:25.025621 13649 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.022498 13566 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:24.980875 13691 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.021492 13713 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:05:25.036091 13156 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.000s
I20260709 13:05:25.043632  9198 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:25.063892  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:25.035270 13524 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:25.079136 13566 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.046589 13646 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.085141 13677 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.086027  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:25.087258 13690 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:25.089296  9854 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:05:25.089926 13697 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.674s	user 0.004s	sys 0.000s
W20260709 13:05:25.094245 13672 consensus_meta.cc:287] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.719s	user 0.009s	sys 0.002s
I20260709 13:05:25.128059 13672 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.130719 13672 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.114190 13664 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:25.131625 13664 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.132062 13664 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:25.133606  9191 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:25.137444  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:25.145844  9119 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:05:25.146736  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:25.147392  9117 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
W20260709 13:05:25.147948 13521 consensus_meta.cc:287] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.002s
I20260709 13:05:25.148406 13521 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:25.154477 13683 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:25.155223 13683 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:25.030566 13371 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:25.155793 13683 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:25.163473  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:25.164904 13624 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:25.165563 13624 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.167065 13696 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.092370  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2444) took 1671 ms. Trace:
I20260709 13:05:25.095949 13726 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.096694 13336 raft_consensus.cc:3055] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:05:25.098001 13685 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.794s	user 0.010s	sys 0.000s
I20260709 13:05:25.098836 13594 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:25.105856 13650 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.046764 13462 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.012189 13622 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.124639  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:25.125044 13471 consensus_meta.cc:287] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.839s	user 0.010s	sys 0.001s
I20260709 13:05:25.168351 13691 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.167474 13504 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:25.172482 13712 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:25.173967 13674 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:25.175948 13566 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:25.177012  9265 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:25.177608  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:25.178397 13718 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.106s	user 0.002s	sys 0.000s
W20260709 13:05:25.179252 13719 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:05:25.181742 13521 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:25.181948 13692 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.103322 13668 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:25.183125 13698 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:25.182763 13362 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.184607 13646 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.447875  9235 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:25.449246  9235 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:25.184837  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2186) took 1569 ms. Trace:
I20260709 13:05:25.188293  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:25.185994 13708 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.111362 13156 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.093214 13713 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:25.114043 13524 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.191115 13274 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:25.167065 13579 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:25.192421 13528 raft_consensus.cc:3055] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:25.193605 13666 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:25.194453 13664 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.197564  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:25.139266  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:25.090466 13701 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.204653  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:25.205785  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:25.494283  9860 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:25.224112  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 223650us
I20260709 13:05:25.200062  9118 rpcz_store.cc:276] 0709 13:05:23.420894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.423127 (+  2233us) service_pool.cc:224] Handling call
0709 13:05:23.423511 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:23.438496 (+ 14985us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:05:23.989620 (+551124us) log.cc:844] Serialized 14 byte log entry
0709 13:05:24.395831 (+406211us) log.cc:844] Serialized 14 byte log entry
0709 13:05:24.396161 (+   330us) log.cc:844] Serialized 14 byte log entry
0709 13:05:24.447122 (+ 50961us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:24.447187 (+    65us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:24.447253 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.509699 (+ 62446us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b180045e6e0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:25.029036 (+519337us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b48000d0a58. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:05:25.092160 (+ 63124us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b5400286254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:25.092203 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:25.092314 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":431640,"prepare.queue_time_us":512447,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":3291,"raft.queue_time_us":121120,"raft.run_cpu_time_us":11544,"raft.run_wall_time_us":619478,"spinlock_wait_cycles":83208448,"thread_start_us":554061,"threads_started":2,"wal-append.queue_time_us":510388}
I20260709 13:05:25.226137  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.270673  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:25.271502  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:25.301409  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.308480  9859 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:25.496882  9117 raft_consensus.cc:2410] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:25.309900  9192 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 13:05:25.498294  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.499691  9114 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:25.446514 13476 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.217658 13336 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:25.219018 13672 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:25.387029  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 64210.1us
I20260709 13:05:25.452916 13596 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:25.526844  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:25.223140 13478 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:05:25.387892  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.387723  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.461943 13521 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:25.452085  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2657) took 1345 ms. Trace:
I20260709 13:05:25.538475  9267 rpcz_store.cc:276] 0709 13:05:24.106492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.133021 (+ 26529us) service_pool.cc:224] Handling call
0709 13:05:24.160468 (+ 27447us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:24.193321 (+ 32853us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:24.193353 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:24.227585 (+ 34232us) log.cc:844] Serialized 52 byte log entry
0709 13:05:24.378136 (+150551us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:24.414381 (+ 36245us) log.cc:844] Serialized 14 byte log entry
0709 13:05:24.414729 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 13:05:24.414884 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.414960 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:24.521151 (+106191us) raft_consensus.cc:1685] finished
0709 13:05:24.521180 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:25.170928 (+649748us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b18004074a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:25.451914 (+280986us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b54002a2e50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:25.452029 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40083,"prepare.queue_time_us":86059,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1925,"raft.queue_time_us":58898,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":32689,"spinlock_wait_cycles":91882624,"thread_start_us":149210,"threads_started":1,"wal-append.queue_time_us":202610,"wal-append.run_cpu_time_us":2579,"wal-append.run_wall_time_us":1080508}
I20260709 13:05:25.453884  9115 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:25.313001 13526 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.454479  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:25.542539 13526 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.543267  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.543519 13668 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:25.544008  9269 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:25.544847 13526 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.545112  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.378573 13493 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:25.546041 13493 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.546968 13170 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:25.547083  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.547503 13170 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:25.220444 13685 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.206830  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:25.548381  9857 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:25.499581  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:25.377959  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:25.550953  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:25.500085  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.312011  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1290) took 1123 ms. Trace:
W20260709 13:05:25.506955  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 367779us
I20260709 13:05:25.552390  9856 rpcz_store.cc:276] 0709 13:05:24.188420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.188565 (+   145us) service_pool.cc:224] Handling call
0709 13:05:24.194262 (+  5697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.418494 (+224232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.311765 (+893271us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54002e9d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:25.311948 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396159744}
I20260709 13:05:25.446965 13595 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:25.454900 13371 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.530296  9825 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:25.451078  9198 rpcz_store.cc:276] 0709 13:05:23.614994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.623354 (+  8360us) service_pool.cc:224] Handling call
0709 13:05:25.184681 (+1561327us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400398854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:25.184783 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3132333824}
I20260709 13:05:25.533703  9189 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:25.452155  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:25.541208 13708 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.546317  9161 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:25.462863  9855 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:05:25.525797 13659 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:25.446632 13653 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:05:25.551563  9265 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:25.552078  9270 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:25.581351  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:25.551795  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.223649 13726 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:25.494215 13528 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:25.582605  9267 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:25.465967 13725 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.495952 13624 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:25.568813  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.566789  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.592212  9268 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:25.559818  9123 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: []
W20260709 13:05:25.566753  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:25.573662  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:25.393414 13622 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:25.556625 13170 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:25.591192  9264 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:25.602826  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.582569  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.605594  9270 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:25.389341 13712 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.571619 13646 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.584276  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.584416  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:25.536612 13648 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:05:25.625692  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.624913  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.627982  9198 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:25.628998  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:25.629681  9198 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:25.630249  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.585964  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.578459 13274 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:25.532354 13476 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.637362  9189 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:25.638270  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.583822  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:25.644380  9270 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:25.621765  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:25.646018  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.537712 13672 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:25.609086 13372 consensus_meta.cc:287] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.656s	user 0.005s	sys 0.005s
I20260709 13:05:25.553129 13596 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.594184 13471 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:25.649479  9189 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:25.611920 13624 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:25.653167  9857 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:25.565850 13595 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.648933  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.652150  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:25.655560  9198 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:25.675483  9265 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
W20260709 13:05:25.624919 13298 consensus_meta.cc:287] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.001s
I20260709 13:05:25.649212  9860 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
W20260709 13:05:25.670536  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 219838us
I20260709 13:05:25.551393 13478 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.663297  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.677975  9268 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:25.682838  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 809) took 1482 ms. Trace:
I20260709 13:05:25.623288 13659 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.680225  9854 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 13:05:25.682993  9189 raft_consensus.cc:2410] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:05:25.679626  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:25.684721  9860 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:25.581579 13371 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:25.658571 13718 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:05:25.692602 13708 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.693320 13708 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.689270 13653 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:25.685563  9192 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:05:25.694589 13653 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.695235  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1103) took 1057 ms. Trace:
I20260709 13:05:25.667053 13696 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:25.695984 13596 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.685640  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:25.683231  9122 rpcz_store.cc:276] 0709 13:05:24.200828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.201000 (+   172us) service_pool.cc:224] Handling call
0709 13:05:25.682666 (+1481666us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400254254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:25.682786 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376667520}
I20260709 13:05:25.688098  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:25.698144  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:25.678591  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.695688 13708 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.686776  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:25.690117  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:25.647012 13493 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.699631  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:25.705076  9189 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:25.695662  9861 rpcz_store.cc:276] 0709 13:05:24.637376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.641792 (+  4416us) service_pool.cc:224] Handling call
0709 13:05:24.757455 (+115663us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:24.757838 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:24.757864 (+    26us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:24.781662 (+ 23798us) log.cc:844] Serialized 78 byte log entry
0709 13:05:24.973023 (+191361us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:25.035289 (+ 62266us) log.cc:844] Serialized 14 byte log entry
0709 13:05:25.036099 (+   810us) log.cc:844] Serialized 14 byte log entry
0709 13:05:25.038255 (+  2156us) log.cc:844] Serialized 14 byte log entry
0709 13:05:25.145069 (+106814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.145155 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:25.177704 (+ 32549us) spinlock_profiling.cc:243] Waited 47.2 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:25.694981 (+517277us) raft_consensus.cc:1685] finished
0709 13:05:25.695004 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:25.695171 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1687,"prepare.queue_time_us":50239,"prepare.run_wall_time_us":36,"raft.queue_time_us":89951,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":19177,"spinlock_wait_cycles":130624384,"thread_start_us":325999,"threads_started":3,"wal-append.queue_time_us":265624}
I20260709 13:05:25.737656  9268 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:25.559428 13728 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.156s
I20260709 13:05:25.601480 13725 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.670802 13372 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:25.679545 13720 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:05:25.705729 13171 consensus_meta.cc:287] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.665s	user 0.008s	sys 0.001s
I20260709 13:05:25.700306  9198 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:25.770530  9198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:25.771214  9198 raft_consensus.cc:606] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:25.771606  9198 raft_consensus.cc:614] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:25.610486 13336 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.756167  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:25.646721 13646 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.757681  9860 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:25.702622 13462 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:25.705693 13650 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:25.760251  9857 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:25.784029  9860 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:25.784746  9857 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:25.763193  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:25.764621  9269 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:25.766953  9268 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:25.765993  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 168871us
I20260709 13:05:25.768253  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:25.757300 13653 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.774946  9862 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:25.689908 13298 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.777215  9198 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:05:25.704576 13659 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:25.760672  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 274490us
I20260709 13:05:25.764990 13674 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:25.786965  9857 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:25.787510  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:25.820284 13657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.000s	sys 0.012s
W20260709 13:05:25.837024  9853 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1283) took 1854 ms (client timeout 1784 ms). Trace:
I20260709 13:05:25.778373 13156 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:25.790339 13466 consensus_meta.cc:287] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.753s	user 0.010s	sys 0.000s
W20260709 13:05:25.837466  9853 rpcz_store.cc:269] 0709 13:05:23.983196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:23.983306 (+   110us) service_pool.cc:224] Handling call
0709 13:05:25.836777 (+1853471us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b540070e4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:25.836952 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1046670592}
I20260709 13:05:25.838413  9858 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:05:25.773929 13471 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:25.788260 13650 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:25.847146 13735 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
W20260709 13:05:25.848054 13737 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:05:25.848230 13736 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:05:25.848801 13738 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:05:25.848892  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 61453.5us
I20260709 13:05:25.849890 13712 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:25.789750 13524 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.784447 13701 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:25.752743 13528 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:25.772780 13171 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:25.861114 13730 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:05:25.826336 13664 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.834942 13362 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.840993 13372 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:25.886880  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2071) took 1671 ms. Trace:
I20260709 13:05:25.887432  9263 rpcz_store.cc:276] 0709 13:05:24.215735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.216886 (+  1151us) service_pool.cc:224] Handling call
0709 13:05:24.217220 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.538256 (+321036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.886537 (+1348281us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400303ed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:25.886822 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657297280}
I20260709 13:05:25.854161 13650 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:25.852727 13466 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:25.853727 13685 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:25.853526 13653 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:25.872536 13686 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:25.838697 13657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.000s	sys 0.013s
I20260709 13:05:25.899547  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:25.903479  9190 raft_consensus.cc:606] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:25.903668  9120 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:05:25.903986  9190 raft_consensus.cc:614] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:25.895296  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 243213us
I20260709 13:05:25.994709  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:25.978685  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 382440us
I20260709 13:05:25.997644  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:25.998216  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2665) took 1463 ms. Trace:
I20260709 13:05:25.998800  9266 rpcz_store.cc:276] 0709 13:05:24.534683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.539155 (+  4472us) service_pool.cc:224] Handling call
0709 13:05:24.540525 (+  1370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:24.769029 (+228504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.904996 (+1135967us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54001c61d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:25.998136 (+ 93140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523402496}
I20260709 13:05:25.999250  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:26.001663  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:25.853956 13471 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:26.055495  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.054049  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 52482.5us
I20260709 13:05:25.896255 13528 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:05:26.076689  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 80099.9us
I20260709 13:05:25.886405 13692 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.088568  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:26.093086  9825 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:26.089002  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.089257  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:26.091449  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.100193  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:26.077677  9020 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 14, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 14 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:26.101432  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.105453  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:26.107003  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:26.110978  9853 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:26.112800  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:26.115089  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:26.117455  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.124228  9857 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:26.128229 13471 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.129347 13298 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:26.086233 13476 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.130342  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 846) took 1263 ms. Trace:
I20260709 13:05:26.130723  9194 rpcz_store.cc:276] 0709 13:05:24.866413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.866611 (+   198us) service_pool.cc:224] Handling call
0709 13:05:24.867044 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:24.911786 (+ 44742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:24.911833 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:24.912394 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:05:25.311601 (+399207us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:25.312224 (+   623us) log.cc:844] Serialized 14 byte log entry
0709 13:05:25.386903 (+ 74679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.566884 (+179981us) spinlock_profiling.cc:243] Waited 44.2 ms on lock 0x7b5400399c54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:25.566940 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:26.129295 (+562355us) spinlock_profiling.cc:243] Waited 44.4 ms on lock 0x7b48000030d8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:05:26.129745 (+   450us) raft_consensus.cc:1685] finished
0709 13:05:26.129767 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.129885 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74455,"prepare.queue_time_us":184,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":86923,"raft.run_cpu_time_us":12743,"raft.run_wall_time_us":742099,"spinlock_wait_cycles":203657216,"thread_start_us":382524,"threads_started":1,"wal-append.queue_time_us":760640}
I20260709 13:05:26.134984 13682 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:26.137660 13716 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 1.129s	user 0.005s	sys 0.011s
I20260709 13:05:26.140825 13171 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:26.142963 13659 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.143714  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:26.144189 13748 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:05:26.075857 13674 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:26.144846  9120 raft_consensus.cc:2410] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:05:26.145058 13747 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.145593 13747 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:26.087294 13595 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:26.091138 13717 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:05:26.151186  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:26.157045  9119 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:26.158116  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.159446  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:26.053994 13672 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.077934 13728 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:26.161015  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:25.901098 13372 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.055557 13466 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:26.162851  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:26.116056 13684 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:26.167346  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.085773 13720 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:26.076534 13700 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:26.162745  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2434) took 1046 ms. Trace:
I20260709 13:05:26.173266  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:26.102679 13462 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:26.115833 13712 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:26.155181 13478 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.177527  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.180998  9119 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:26.195410  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:26.164759 13298 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.199069 13744 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:26.199424 13298 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.184206  9121 rpcz_store.cc:276] 0709 13:05:25.116756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.151951 (+ 35195us) service_pool.cc:224] Handling call
0709 13:05:25.152351 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:25.152795 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:25.152853 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:25.153415 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:05:25.395241 (+241826us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:25.395303 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:25.395420 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:26.162547 (+767127us) raft_consensus.cc:1685] finished
0709 13:05:26.162567 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.162693 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17514,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":69,"thread_start_us":241351,"threads_started":1,"wal-append.queue_time_us":404053}
I20260709 13:05:26.199715 13744 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.206120  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.206794  9119 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:26.162233 13747 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:26.098946 13691 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:05:26.204044 13725 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:26.208115  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:26.205514 13646 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.208506 13691 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:26.208751  9119 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:26.168526 13728 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:26.209115 13723 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.137s	user 0.007s	sys 0.018s
I20260709 13:05:26.209515 13728 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:26.210796 13170 consensus_meta.cc:287] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.000s
I20260709 13:05:26.211182 13170 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:26.211503  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 838) took 1396 ms. Trace:
I20260709 13:05:26.198555  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.211920  9196 rpcz_store.cc:276] 0709 13:05:24.814961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.815103 (+   142us) service_pool.cc:224] Handling call
0709 13:05:24.815440 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:24.815806 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:24.815833 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:24.832229 (+ 16396us) log.cc:844] Serialized 52 byte log entry
0709 13:05:24.948505 (+116276us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:24.948550 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:24.948625 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:26.208436 (+1259811us) spinlock_profiling.cc:243] Waited 40.3 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:26.211214 (+  2778us) raft_consensus.cc:1685] finished
0709 13:05:26.211233 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.211448 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1817,"raft.queue_time_us":574,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":5398,"spinlock_wait_cycles":92669184,"thread_start_us":67992,"threads_started":1,"wal-append.queue_time_us":166201}
I20260709 13:05:26.209175 13744 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.213862  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.196916  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:26.219095  9119 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:05:26.168032 13674 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:26.218515  9235 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:26.208958 13691 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:26.222622  9089 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:26.175299 13371 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.223026  9853 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:26.225834  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:26.180647 13684 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:26.228087 13170 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:26.228245 13646 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:26.229229  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:26.195144 13677 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:26.184619 13720 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:26.242519 13372 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:26.223091 13595 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:26.230680 13744 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:26.248492  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:26.248932  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.222440 13679 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.228907 13597 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:26.231076 13593 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.197633 13700 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.216149 13664 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:05:26.304327  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:26.251085  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.306633  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:26.250746  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:26.319839  9862 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:26.319873  9121 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:26.244102 13720 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:26.323570  9191 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:05:26.325001  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:26.243772 13595 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:26.329866  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 79209.9us
I20260709 13:05:26.334908  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:26.335639  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:26.298424 13646 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:26.251233 13677 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:26.370523  9121 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:26.224354 13691 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.316097 13597 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.325877 13371 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.329483 13538 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:26.368072 13171 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:26.329629 13362 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:26.367069 13595 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:26.327800 13744 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:26.366868 13708 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.371452 13372 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:26.386320 13524 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:26.326174 13593 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.398352 13597 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:26.328346 13679 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:26.414801 13754 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:05:26.407048 13691 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:26.404044 13538 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:26.331827 13684 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.419929 13677 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:26.421849 13700 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:26.422425 13538 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:26.412027 13524 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:26.606645  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1112) took 1521 ms. Trace:
I20260709 13:05:26.404917 13171 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.607105 13504 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:26.407330 13362 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.607307  9855 rpcz_store.cc:276] 0709 13:05:25.085607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.085750 (+   143us) service_pool.cc:224] Handling call
0709 13:05:26.606179 (+1520429us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540065fed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:26.606404 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799627520}
I20260709 13:05:26.608413 13504 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.608531 13524 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.609257  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:26.610431  9855 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:26.610710 13362 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.403487 13371 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:26.410072 13372 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.489810  9020 catalog_manager.cc:5697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 7, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 7 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:05:26.437228 13597 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.508191  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2459) took 1362 ms. Trace:
I20260709 13:05:26.507205  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:26.512897  9021 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36610 (request call id 326) took 26081 ms (client timeout 15000 ms). Trace:
I20260709 13:05:26.513088  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:26.516675  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:26.520659  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2203) took 2136 ms. Trace:
I20260709 13:05:26.521144  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2210) took 1215 ms. Trace:
W20260709 13:05:26.534804  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:26.552362  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 221212us
I20260709 13:05:26.555372  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 2087) took 1089 ms. Trace:
I20260709 13:05:26.557318  9238 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:26.560072  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:26.561695  9263 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:26.562510  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:26.564141  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:26.565092  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:26.566082  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:26.566712  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2080) took 1171 ms. Trace:
I20260709 13:05:26.568012  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:26.598496  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.602564  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:26.604377  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1115) took 1116 ms. Trace:
I20260709 13:05:26.604858  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:26.605484  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1104) took 1963 ms. Trace:
I20260709 13:05:26.605638  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:26.502735  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2433) took 1645 ms. Trace:
W20260709 13:05:26.448737 13601 consensus_meta.cc:287] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.414s	user 0.012s	sys 0.000s
W20260709 13:05:26.437215 13622 consensus_meta.cc:287] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.821s	user 0.004s	sys 0.005s
I20260709 13:05:26.797088 13377 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:26.438438 13593 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.799412  9861 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:05:26.800417  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 198313us
I20260709 13:05:26.800978  9272 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
W20260709 13:05:26.441603 13650 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.539s	user 0.008s	sys 0.000s
I20260709 13:05:26.630515 13362 consensus_queue.cc:1059] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.174s
I20260709 13:05:26.804569 13650 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:26.807394  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.808712 13712 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:26.809461 13712 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.810091 13712 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:26.809689  9020 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:26.613409 13471 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.811973 13277 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.812220 13471 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.812430 13601 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:26.813318 13700 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:26.816255 13471 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.403916 13674 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.816154 13725 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:26.819033 13725 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:26.623608 13257 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:26.821985 13257 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.429171 13679 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:26.426962 13683 consensus_meta.cc:287] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 1.271s	user 0.007s	sys 0.005s
I20260709 13:05:26.824539  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:26.825209 13257 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.826000  9853 raft_consensus.cc:2410] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:26.423696 13684 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:26.834509  9857 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:26.410477 13708 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:26.431025 13713 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.936s	user 0.007s	sys 0.000s
W20260709 13:05:26.441615 13714 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.422s	user 0.001s	sys 0.015s
W20260709 13:05:26.435549 13724 log.cc:927] Time spent T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 1.364s	user 0.009s	sys 0.011s
I20260709 13:05:26.453080 13726 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:26.431038 13728 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.916244 13728 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:26.438534 13732 log.cc:927] Time spent T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.742s	user 0.002s	sys 0.000s
I20260709 13:05:26.918151  9122 raft_consensus.cc:2368] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11.
I20260709 13:05:26.409405 13744 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.921008 13744 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:26.444917 13756 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:05:26.453369 13595 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.611480  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:26.924441  9855 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:26.612809  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1298) took 1425 ms. Trace:
I20260709 13:05:26.754956  9269 rpcz_store.cc:276] 0709 13:05:25.466104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.544841 (+ 78737us) service_pool.cc:224] Handling call
0709 13:05:26.554898 (+1010057us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54000f9bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:26.555130 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503201536}
I20260709 13:05:26.755792  9114 rpcz_store.cc:276] 0709 13:05:24.857408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.857804 (+   396us) service_pool.cc:224] Handling call
0709 13:05:24.881710 (+ 23906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.500123 (+618413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.502061 (+1001938us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b54001e3f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:26.502513 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420823424}
I20260709 13:05:26.925676 13595 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.754720  9195 rpcz_store.cc:276] 0709 13:05:25.305835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.306363 (+   528us) service_pool.cc:224] Handling call
0709 13:05:25.306634 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.744651 (+438017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:25.744682 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:25.744740 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:25.744776 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.520578 (+775802us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5400417dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:26.520648 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.520925 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1006854912}
I20260709 13:05:26.679919  9118 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:26.755493  9115 rpcz_store.cc:276] 0709 13:05:25.145625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.146524 (+   899us) service_pool.cc:224] Handling call
0709 13:05:25.146826 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.542080 (+395254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.503551 (+961471us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54001e3a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:26.508011 (+  4460us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705839872}
I20260709 13:05:26.611392 13504 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.620611 13170 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.756013  9270 rpcz_store.cc:276] 0709 13:05:25.394944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.654289 (+259345us) service_pool.cc:224] Handling call
0709 13:05:25.654552 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.697351 (+ 42799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:25.697389 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:25.697431 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:25.697466 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.566156 (+868690us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b54000eabd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:26.566235 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.566522 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97969920}
I20260709 13:05:26.927305 13170 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:26.755463  9856 rpcz_store.cc:276] 0709 13:05:25.487268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.553741 (+ 66473us) service_pool.cc:224] Handling call
0709 13:05:25.557157 (+  3416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:25.557401 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:25.557432 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:25.566441 (+  9009us) log.cc:844] Serialized 26 byte log entry
0709 13:05:26.040561 (+474120us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:26.040619 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.040713 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:26.603641 (+562928us) raft_consensus.cc:1685] finished
0709 13:05:26.603688 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.604083 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":417918,"threads_started":1,"wal-append.queue_time_us":631722}
I20260709 13:05:26.929900 13170 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:26.795001  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.812722 13277 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.820664 13725 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.932502 13725 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:26.432309 13686 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:26.936165 13686 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:26.454120 13720 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.828162  9271 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:26.828608 13171 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.829337 13362 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:26.938107 13362 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:26.827455  9121 raft_consensus.cc:2368] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11.
I20260709 13:05:26.829898 13371 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:26.830581 13579 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:26.828171  9116 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:26.771416  9858 rpcz_store.cc:276] 0709 13:05:24.642247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.642358 (+   111us) service_pool.cc:224] Handling call
0709 13:05:25.838167 (+1195809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:25.844485 (+  6318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.604959 (+760474us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540070e4d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:26.605259 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2485826816}
I20260709 13:05:26.633894 13331 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.941886 13331 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:26.676337 13682 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.944177 13682 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.946696 13682 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.825685 13683 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:26.772516  9021 rpcz_store.cc:269] 0709 13:05:00.430901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:00.431219 (+   318us) service_pool.cc:224] Handling call
0709 13:05:26.512572 (+26081353us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:18.171878 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:18.172633 (+   755us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:18.172693 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:05:18.172714 (+    21us) tablet.cc:662] Decoding operations
0709 13:05:18.332797 (+160083us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:18.350992 (+ 18195us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:18.355755 (+  4763us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:05:18.693141 (+337386us) tablet.cc:701] Row locks acquired
0709 13:05:18.693203 (+    62us) write_op.cc:260] PREPARE: finished
0709 13:05:18.693455 (+   252us) write_op.cc:270] Start()
0709 13:05:18.693662 (+   207us) write_op.cc:276] Timestamp: P: 1783602318693365 usec, L: 0
0709 13:05:18.693718 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 13:05:18.694573 (+   855us) log.cc:844] Serialized 8146 byte log entry
0709 13:05:19.532526 (+837953us) op_driver.cc:464] REPLICATION: finished
0709 13:05:19.918315 (+385789us) write_op.cc:301] APPLY: starting
0709 13:05:19.925265 (+  6950us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:20.842183 (+916918us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:20.842374 (+   191us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:21.246582 (+404208us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:21.246965 (+   383us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:05:21.247030 (+    65us) write_op.cc:312] APPLY: finished
0709 13:05:21.247481 (+   451us) log.cc:844] Serialized 180 byte log entry
0709 13:05:21.480472 (+232991us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:21.507126 (+ 26654us) write_op.cc:454] Released schema lock
0709 13:05:21.529388 (+ 22262us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:21.616164 (+ 86776us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:21.620795 (+  4631us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1619207,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":257803,"apply.run_cpu_time_us":92104,"apply.run_wall_time_us":1685868,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87767,"num_ops":20,"prepare.queue_time_us":288047,"prepare.run_cpu_time_us":69756,"prepare.run_wall_time_us":897221,"raft.queue_time_us":195897,"raft.run_cpu_time_us":36873,"raft.run_wall_time_us":210623,"replication_time_us":807641,"thread_start_us":827045,"threads_started":5,"wal-append.queue_time_us":639950,"wal-append.run_cpu_time_us":29451,"wal-append.run_wall_time_us":2311430}]]}
W20260709 13:05:26.838963  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 57790.4us
I20260709 13:05:26.637118 13336 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:26.842257 13601 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:26.844141  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.641824 13466 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:26.620174 13156 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:26.847405  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 331158us
I20260709 13:05:26.848948  9264 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:26.851315  9854 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:26.615078  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:26.844496 13528 consensus_meta.cc:287] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.751s	user 0.015s	sys 0.005s
I20260709 13:05:26.939384 13720 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.674423 13659 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.184s
I20260709 13:05:26.857283 13624 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:26.848297  9020 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:26.899436 13761 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:26.871873  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:05:26.886145  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 203973us
I20260709 13:05:26.881680 13650 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:26.897102  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:26.946775 13708 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:26.900723  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 343845us
I20260709 13:05:26.900622 13478 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:05:26.903357 13765 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:26.970041  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:26.970254 13478 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:26.970836 13478 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:26.972087 13478 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.925284  9859 rpcz_store.cc:276] 0709 13:05:25.187445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.187571 (+   126us) service_pool.cc:224] Handling call
0709 13:05:25.193053 (+  5482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:25.497100 (+304047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.612231 (+1115131us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400660b54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:26.612596 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264800128}
I20260709 13:05:26.939314 13362 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.630416 13298 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:26.698282 13538 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.153s
I20260709 13:05:26.960764  9266 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:26.964484 13156 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.968653 13466 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.968369  9020 catalog_manager.cc:5697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 6 to 9, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 9 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:26.969724  9862 raft_consensus.cc:2410] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:26.912482  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 1249) took 1290 ms. Trace:
I20260709 13:05:26.970896  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1118) took 1469 ms. Trace:
I20260709 13:05:26.912154 13713 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.975859  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2456) took 1831 ms. Trace:
I20260709 13:05:26.755033  9193 rpcz_store.cc:276] 0709 13:05:24.385142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:24.394295 (+  9153us) service_pool.cc:224] Handling call
0709 13:05:24.430878 (+ 36583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:26.192390 (+1761512us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b54000503a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f765665f858 00007f76566e3e9d 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c
0709 13:05:26.192448 (+    58us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:26.192473 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:26.192519 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:26.192562 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.520161 (+327599us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b54003b47d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:26.520234 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.520451 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79994,"raft.queue_time_us":81109,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":22166,"spinlock_wait_cycles":2191597056}
I20260709 13:05:26.974278 13478 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:26.977952  9861 rpcz_store.cc:276] 0709 13:05:25.621716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.757900 (+136184us) service_pool.cc:224] Handling call
0709 13:05:26.898882 (+1140982us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54006e9e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:26.912420 (+ 13538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2327568128}
I20260709 13:05:26.987887  9198 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:05:26.965286 13579 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:26.970222 13597 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:26.980602  9116 rpcz_store.cc:276] 0709 13:05:25.144716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.146028 (+  1312us) service_pool.cc:224] Handling call
0709 13:05:25.146350 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:26.940895 (+1794545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.974361 (+ 33466us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540037f5d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:26.975717 (+  1356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3866926464}
I20260709 13:05:26.970417 13765 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:26.995496  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:26.993534  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 198782us
I20260709 13:05:27.000136  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:26.997895  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 95177.1us
I20260709 13:05:26.979902 13331 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:27.050904  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2213) took 1373 ms. Trace:
W20260709 13:05:27.006989  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 138511us
I20260709 13:05:27.007511 13156 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.011931 13685 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:27.051295  9198 rpcz_store.cc:276] 0709 13:05:25.677604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.822939 (+145335us) service_pool.cc:224] Handling call
0709 13:05:25.823252 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:26.988451 (+1165199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:27.050606 (+ 62155us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000677d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:27.050827 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2588583168}
I20260709 13:05:27.052023 13685 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:27.052129  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:27.052562 13685 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.007513 13761 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:27.055449 13772 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.084s	user 0.003s	sys 0.000s
I20260709 13:05:27.021402 13725 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:27.023893 13579 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.025005  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.061975  9120 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:27.029196 13466 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.034590  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.035014  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.063848  9853 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.036031 13528 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:27.036806 13372 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:27.068092  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:27.068292 13156 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:27.069125  9265 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:27.069373  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:27.037137  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 59200.1us
I20260709 13:05:27.070124  9266 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:27.068131 13528 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:27.037720 13170 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:27.037875  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.071939  9862 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.071588 13528 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.073132 13528 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:27.004562 13683 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.063512 13579 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:27.039511 13726 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.075371 13726 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.968684 13624 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.077870 13726 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.041424 13713 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:27.000696 13765 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:27.042762  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 51101us
I20260709 13:05:27.082748  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:27.083483 13624 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:27.084635 13597 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:27.085103 13597 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.085449 13597 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:27.043347 13678 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:27.043419  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.087350 13170 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:27.094974 13726 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:27.047971  9861 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 12 LEARNER]: Advancing to term 13
I20260709 13:05:27.048920  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:27.096370  9267 raft_consensus.cc:2368] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9.
I20260709 13:05:27.095779 13726 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.098956 13726 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:27.050104 13232 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:27.100153 13232 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.101305  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 819) took 1011 ms. Trace:
I20260709 13:05:27.101409 13232 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:26.978075  9860 rpcz_store.cc:276] 0709 13:05:25.488033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:25.785037 (+297004us) service_pool.cc:224] Handling call
0709 13:05:25.785681 (+   644us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:25.786074 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:25.786103 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:25.822210 (+ 36107us) log.cc:844] Serialized 52 byte log entry
0709 13:05:26.052978 (+230768us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:26.215412 (+162434us) log.cc:844] Serialized 14 byte log entry
0709 13:05:26.216178 (+   766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:26.216247 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:26.444204 (+227957us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:26.951291 (+507087us) raft_consensus.cc:1685] finished
0709 13:05:26.956899 (+  5608us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:26.957133 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":145896,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":5991,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":39918,"spinlock_wait_cycles":14865152,"thread_start_us":293365,"threads_started":2,"wal-append.queue_time_us":401489}
I20260709 13:05:27.056708 13761 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:27.043083 13277 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:27.060268 13371 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:27.063081 13466 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:27.095314 13170 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.105051 13466 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.106343 13371 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:27.106765 13371 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:27.054457  9164 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:27.107074 13466 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.066193  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:27.107484  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 53120us
I20260709 13:05:27.107862 13371 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.088919  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:27.089493 13156 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.112141  9161 leader_election.cc:400] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11.
I20260709 13:05:27.112730 13156 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.045533  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:27.092162 13682 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:27.092928 13747 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:27.115967 13747 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:27.116369 13747 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.116680  9020 catalog_manager.cc:5697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.117442 13747 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.094070 13684 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:27.118940 13684 consensus_queue.cc:1243] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:27.119506 13684 raft_consensus.cc:3055] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:27.104429 13678 consensus_queue.cc:1059] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:05:27.119899 13684 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.120939 13684 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.122380 13684 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.101754  9123 rpcz_store.cc:276] 0709 13:05:26.089936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:26.117244 (+ 27308us) service_pool.cc:224] Handling call
0709 13:05:27.100973 (+983729us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b5400285354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:27.101147 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2258430720}
I20260709 13:05:27.084460 13538 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:27.059381 13362 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.128263 13371 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.067543  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:27.108476 13683 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:27.145422 13683 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.146553 13683 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.147493 13538 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:27.148058 13747 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.148373 13538 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.148391 13336 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:27.152586 13336 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.153393  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.154238  9860 raft_consensus.cc:2368] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
I20260709 13:05:27.155071  9861 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 13 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:05:27.155504  9861 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 7
I20260709 13:05:27.155819  9861 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:05:27.156133  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.156308  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.157083  9119 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 4.
I20260709 13:05:27.157218 13674 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:27.047197 13712 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.132154 13682 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:27.158023 13682 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.158537 13682 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:27.137432 13678 raft_consensus.cc:3055] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:27.159945 13678 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:27.138072  9164 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:27.142436  9264 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:27.118834  9020 catalog_manager.cc:5697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:27.132491  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 43694.8us
I20260709 13:05:27.167582 13478 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.169752  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:27.169873 13685 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.156219  9861 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 13 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:27.157599 13674 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.107075 13170 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.171561 13674 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:27.172559  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:27.171838 13331 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:27.173137 13331 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:27.173609 13331 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:27.174830  9853 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:27.134168  9859 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
W20260709 13:05:27.179256  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.190222 13686 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.191710  9270 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:27.197463  9862 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:27.199163 13170 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:27.199579 13170 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.199982 13170 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.157332  9089 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:27.165126  9020 catalog_manager.cc:5697] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.207492  9162 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:27.201269  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:27.215854  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 77941.4us
I20260709 13:05:27.217509  9162 leader_election.cc:400] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9.
I20260709 13:05:27.218005  9162 leader_election.cc:403] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:27.197816 13712 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:27.220700  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:27.221755 13538 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:27.226643  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.216622 13659 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:27.214628 13685 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.236342  9020 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 9, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 9 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:27.243386  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:27.245229  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.245983  9862 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:27.235028 13504 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.240280 13257 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:27.247514  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.247797 13681 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:27.247885 13257 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.248319 13681 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:27.248759 13681 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.248868 13257 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.250445 13257 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:27.250641  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 93452.4us
I20260709 13:05:27.254709  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.238380 13659 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.259577 13466 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:27.276945 13659 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.277133 13466 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:27.233675 13678 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.260630 13597 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.278378  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.278894 13678 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.279588 13601 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:27.280069 13597 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.267880 13677 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:27.280395 13720 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:27.280727 13677 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:27.281078 13720 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:27.281143 13677 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.280169 13601 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:27.282637 13601 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.280736 13771 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:27.283541 13466 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.284307 13771 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.284729 13771 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.264321 13712 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:27.269677 13336 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.286391 13712 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.271838  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.288136  9118 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:27.273149  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.289175  9120 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:27.273825  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.274575  9116 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:27.274497  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.274781  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.291543  9114 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.291666  9117 raft_consensus.cc:2393] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:27.275714  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.276427 13686 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
W20260709 13:05:27.280020  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59440.7us
I20260709 13:05:27.268539 13765 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.297575  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.297050  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:27.301887  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.305356  9235 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.306530  9122 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:27.308868  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.309544  9116 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:27.310199 13257 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:27.310864 13257 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.312944 13257 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.319168  9117 raft_consensus.cc:2410] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:27.322388 13659 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.324167  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:27.316316  9020 catalog_manager.cc:5697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 11. New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.329834  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.331775  9116 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:27.332574 13685 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:27.333016 13685 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:27.333375 13685 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:27.337411 13765 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:27.337275 13257 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:27.338798 13712 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:27.317101 13336 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:27.340145 13471 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:27.341995 13471 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.342486 13471 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:27.338145 13257 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:27.344112 13257 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.345196 13257 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.346561 13257 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.348021 13371 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.348650 13371 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.350829 13371 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.341991 13336 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:27.352548 13504 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9.
I20260709 13:05:27.353029 13336 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.361573  9020 catalog_manager.cc:5697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 6 to 7, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 7 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:05:27.362967 13677 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.364090  9121 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:27.365098 13677 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.370304  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.372072  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.372485  9121 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:27.374221  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.375480  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.379673  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.379954 13232 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:27.380736 13232 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.339830 13659 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:27.381431 13678 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:27.381101 13597 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:27.382479 13597 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.363461  9020 catalog_manager.cc:5697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 11, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.385490  9114 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:27.391355  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:27.395545  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.397383  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.397317  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.398247  9118 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:27.398355  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.399034  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.399262  9114 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:27.398216  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.402014  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.404338  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:27.382992 13597 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.405284  9263 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:27.407014  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.383010 13232 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:27.359438 13331 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.406435  9115 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:27.412675  9020 catalog_manager.cc:5697] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 12, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 12 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:27.415467  9235 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.371488 13685 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.413458 13371 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:27.421660  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.423300  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.396011 13682 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.388471 13674 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.403687 13471 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.436965  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.438024  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.433588  9235 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.403255 13170 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.403939 13681 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.444003  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:27.456441  9114 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:27.459311  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:27.459975  9189 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:27.405036 13771 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.464030  9269 raft_consensus.cc:1240] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 4. Current term is 5. Ops: []
I20260709 13:05:27.467259  9120 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:27.469966  9123 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:27.471231 13685 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:27.471711  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:27.465058 13232 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:27.472668 13232 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:27.473050 13674 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.448149 13371 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:27.474465 13371 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:27.476874 13784 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:27.436899 13362 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:27.479718 13362 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.480145 13362 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.484484  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:27.485044 13170 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:27.485731  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 54950.3us
I20260709 13:05:27.485932 13685 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:27.486773 13685 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:27.487013  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.487324  9087 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:27.487901  9862 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:27.488004 13712 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:27.487792 13685 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.488533  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:27.494082  9195 raft_consensus.cc:2410] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:27.494987  9193 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:27.496582  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.486177  9826 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:27.500351  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:27.501075  9267 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:27.505203  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 169057us
I20260709 13:05:27.490777  9020 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 6 to 10, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.471572 13682 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.510136  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:27.511579  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.512059  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.508514  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:27.515837  9263 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 16: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:27.473068 13232 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.517288  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.518040  9122 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.523164  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.523380  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.524467  9118 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:27.524276 13659 consensus_queue.cc:1059] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:27.525347 13257 raft_consensus.cc:3055] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:05:27.525800 13257 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.526706 13257 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.527823 13257 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:27.502066 13677 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:27.542979  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.549774  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.550606  9117 raft_consensus.cc:2393] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:27.551314  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:27.551424  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.556294  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.557273  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.557955  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.558691  9853 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:27.560132  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 74040.7us
I20260709 13:05:27.555063  9020 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 10, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.539544 13624 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:27.561123  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.561991  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.562074  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.564002  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.566192  9861 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:27.566497  9859 raft_consensus.cc:2368] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9.
W20260709 13:05:27.566691  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 79509.6us
I20260709 13:05:27.568275  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.563935  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:27.565725  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:27.572225 13331 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.576810  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.577483  9114 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:27.578369  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.580885  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.581534  9859 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 6.
I20260709 13:05:27.585125 13478 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:27.587139  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.579479  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.569423  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:27.593940  9862 raft_consensus.cc:2410] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:27.594427  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:27.595535  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:27.542330 13156 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:05:27.600334  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.560807 13747 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:27.609339  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:27.610736  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 74344.9us
I20260709 13:05:27.611665  9161 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:27.612866  9859 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:27.605608  9197 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:27.607825  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:27.616286  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.588177 13659 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:27.616356  9860 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:27.622486  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:27.624521  9266 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:27.627043  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:27.627686  9263 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:27.633370  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.634023  9860 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:27.636102  9853 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 6
I20260709 13:05:27.636722  9853 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:05:27.638197  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.636054  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.640348  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:27.643343  9853 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 13 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:27.644078  9263 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:27.645249  9861 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:27.653193  9195 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:27.653810  9122 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:27.654556  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.655169  9856 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:27.657075  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:27.662854  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.637462 13624 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:27.666306  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 96597.4us
I20260709 13:05:27.625044  9020 catalog_manager.cc:5697] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 6, term changed from 4 to 6, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added, now has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 6 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:27.666949  9857 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:27.669696  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 58150.8us
I20260709 13:05:27.670380  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:27.652988 13471 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.673837  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:27.674777  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:27.679497  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.680157  9855 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:27.679059  9195 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:27.665776 13681 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.667994 13331 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.688462  9020 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:27.687714  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:27.691051  9197 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:27.691336  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.671381 13788 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:27.701449 13720 consensus_queue.cc:1059] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:27.703857  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:27.707818 13783 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:05:27.652263 13747 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.683357 13771 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:27.722551  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:27.736900  9020 catalog_manager.cc:5697] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 9, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 9 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:27.722908  9860 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:27.683557 13624 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.717618  9859 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:27.687456 13504 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:27.686968 13362 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.737018  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:27.744738  9861 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:27.746129  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:27.746753  9266 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:27.748701 13682 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:27.749141 13682 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.750103 13682 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.751397 13682 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:27.711073 13788 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:27.738950  9858 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:27.741648  9269 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:27.742017  9856 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
W20260709 13:05:27.760969  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.744977  9020 catalog_manager.cc:5697] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 8, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 8 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:27.738412  9855 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
W20260709 13:05:27.759009  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 47059.4us
I20260709 13:05:27.764127  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:27.690296 13712 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:05:27.770648  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:27.738785 13747 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:27.787252  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.794754  9189 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:27.796692  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:27.796639  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 43812.5us
I20260709 13:05:27.710758 13331 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.842463  9267 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:27.843942  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:27.756610 13681 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.738081 13232 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:27.744047 13674 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:05:27.772244 13788 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:27.795944 13504 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:27.804689 13156 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:27.853509 13170 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:27.863080  9861 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:27.929701 13156 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.930996 13170 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.930737 13156 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.932163 13170 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:27.864490  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 100424us
I20260709 13:05:27.934865  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:27.935837 13683 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:27.936990  9161 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:27.880125  9198 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:27.937680 13683 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:27.875794  9020 catalog_manager.cc:5697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:27.939493  9020 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:27.942502  9020 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:27.840912 13747 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:27.944330 13747 raft_consensus.cc:697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:27.941089  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:27.945520 13683 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:27.945335 13747 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:27.876824 13674 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:27.873715  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:27.948483  9189 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
W20260709 13:05:27.949196  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 226527us
W20260709 13:05:27.950891  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 313992us
I20260709 13:05:27.958132  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:27.959875  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:27.976001 13795 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:27.976670 13795 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:27.978174 13789 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 13:05:27.979015 13795 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:27.982041 13791 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:05:27.988080 13787 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
W20260709 13:05:27.988296  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:27.992832  9193 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:05:28.006179  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:28.006757  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:28.007040  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 50465.2us
I20260709 13:05:28.008282  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:28.009061  9195 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:28.009963 13601 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:28.011579  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:28.012102 13232 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:28.012426  9196 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:28.013710  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.014358  9196 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:28.017016  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.017584 13331 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:28.018384  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.019174  9196 raft_consensus.cc:2410] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:05:28.020117  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.021973  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.024235  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:28.024878  9196 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:28.025333  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:27.960757 13579 consensus_queue.cc:1059] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:05:28.026611  9270 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:27.991902 13677 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:27.952667 13681 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.027333 13677 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:28.028962 13681 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:28.028555 13677 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:27.944258  9020 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 12, pending config changed. New cstate: current_term: 12 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:27.970322 13684 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:28.032112 13684 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:28.033022 13684 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:28.033448  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.045590  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:28.046700 13624 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:28.047776  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.048767  9270 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:28.049225 13466 raft_consensus.cc:3055] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:28.049691 13466 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:28.050397  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:28.050668 13466 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:28.051590 13331 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:28.052300 13331 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:28.055258  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:28.056034  9856 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:27.979640 13795 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:28.069847  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:28.070504  9858 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:28.073971  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:28.075248  9115 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:28.068387  9825 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:28.031558 13659 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:28.078881  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:28.080283  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.081537  9196 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:28.072156  9020 catalog_manager.cc:5697] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 6 to 9, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 9 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:28.087256  9860 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 8: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:28.087723  9020 catalog_manager.cc:5697] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 1 to 8, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 8 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:28.090415  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.090830  9120 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:28.091141  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.092252  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:28.093734  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:28.094703  9855 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:28.098187  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:28.099426  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.097441  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:28.074463 13478 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:28.110003  9115 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:28.117223  9858 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:28.119004  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:28.119102  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.119163  9020 catalog_manager.cc:5697] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 2 to 8, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 8 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:28.120654  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 162689us
I20260709 13:05:28.120044  9190 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
W20260709 13:05:28.104880  9826 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.968s (SENT) [suppressed 24 similar messages]
W20260709 13:05:28.127184  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 177277us
I20260709 13:05:28.135833  9858 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:05:28.090056 13371 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:28.091452 13795 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:28.111687 13478 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.089813 13538 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:28.107663 13466 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:28.095373 13232 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:28.129287  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:28.148166  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.122654 13362 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:28.156893  9194 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 5.
I20260709 13:05:28.118865 13504 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 13:05:28.154225 13597 consensus_meta.cc:287] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.002s
I20260709 13:05:28.151599 13478 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.158887 13683 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:28.084379 13331 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:28.084049  9189 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:28.171196  9189 raft_consensus.cc:606] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:28.171535  9189 raft_consensus.cc:614] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:28.152428 13624 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:28.167452  9270 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:28.167294  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.167991  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:28.137632 13336 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:05:28.180022  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.183486  9164 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:28.187080  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 178903us
I20260709 13:05:28.179284  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.188733  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.189216  9269 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:28.148240 13371 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.172139 13504 consensus_queue.cc:1243] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:05:28.195318  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.195977  9192 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:28.196599  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.173313 13712 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:28.163358 13232 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:28.200253  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.201028  9263 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:28.184162 13624 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:28.162432 13538 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.164558 13597 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:28.202747  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:28.207216  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.201905 13712 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:28.212141 13712 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:28.212492  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.212949  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.213129  9189 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:28.193660 13683 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:28.214150  9114 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:28.204653  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:28.211642  9123 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:28.215909  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.216670  9263 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:28.223183  9120 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:28.230813  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:28.234553  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.235414  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.236550  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:28.238036  9192 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:28.238075 13771 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:28.239451 13712 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.241568 13712 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:28.244925  9269 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:28.208037 13624 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:28.247479  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.246748  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:28.257161  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43520.1us
I20260709 13:05:28.248127  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.263845  9189 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:28.248375  9264 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:28.248920  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.249223  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.266361  9271 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:28.249858  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.252167  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.206444 13788 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:28.256613  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.257402  9164 leader_election.cc:400] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 8. Current term is 9.
I20260709 13:05:28.261937  9194 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:28.247776 13597 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:28.272778  9272 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:28.207787 13797 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:28.277231  9263 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.278542  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.280037  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:28.269299  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 45283.3us
I20260709 13:05:28.280843  9268 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:28.281455  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.281785  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.282075  9266 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:28.282487  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.283195  9267 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:28.270210  9123 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:28.284044  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.284255  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.284765  9267 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:28.285331  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.285434  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.286513  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.287206  9267 raft_consensus.cc:2368] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:05:28.272105  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:28.288043 13809 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:05:28.284068  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.204978 13793 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:28.271152  9269 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:28.288316 13683 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:28.293785  9195 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:28.282672  9268 raft_consensus.cc:2410] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:05:28.295548  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:28.296304  9123 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:05:28.296773 13800 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:05:28.289279  9164 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:28.283022  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.282161  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.278976  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:28.291617  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.295383  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.306270  9263 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:28.316532  9265 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:28.305889  9267 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:28.325853  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.325139  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:28.328318  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 49425.4us
I20260709 13:05:28.331293 13812 consensus_queue.cc:1059] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:28.331915 13362 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:28.332393 13812 raft_consensus.cc:3055] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:28.332784 13812 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:28.333604 13812 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.334820 13812 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:28.336862  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.337598  9270 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:28.339605  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:28.340608  9858 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 7: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:28.342059  9164 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:28.344691  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.345707  9266 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:28.350530  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:28.292855 13793 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:28.355790  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:28.356618  9861 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:28.317252 13659 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:28.361575  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:28.348066  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:28.362370  9263 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:05:28.353260  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.222357 13795 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.354527  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:28.357797  9853 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:28.358498  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.351295  9858 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:28.360033  9268 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:28.364784  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.369145  9198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:28.369717  9198 raft_consensus.cc:606] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:28.346418 13797 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:28.367585  9265 raft_consensus.cc:2376] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10. Re-sending same reply.
I20260709 13:05:28.370061  9198 raft_consensus.cc:614] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:28.352396 13682 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:28.362742  9263 pending_rounds.cc:85] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 8
I20260709 13:05:28.376261  9263 pending_rounds.cc:107] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:05:28.376561  9263 raft_consensus.cc:2889] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:28.352131 13684 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:28.371956  9272 raft_consensus.cc:2368] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11.
I20260709 13:05:28.379686  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:28.380296  9860 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:28.381539 13810 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.382069 13810 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.381567  9194 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:28.386655  9859 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:28.387362  9192 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:28.387784  9192 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:28.388615 13808 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:05:28.388535  9192 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.377211  9269 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:28.389200  9859 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:28.391322  9192 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.389591  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:28.393882 13747 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.416188  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:28.416435 13747 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:28.416895  9858 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:28.415917 13805 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:05:28.418555  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:28.419643  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:28.420094  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.421176  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:28.419693  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:28.421761  9120 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:28.422115  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.422957  9269 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:28.359045 13793 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:28.411811  9856 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:28.420956  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:28.433847  9859 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:05:28.431407  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 105631us
I20260709 13:05:28.365721 13683 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:28.372648 13797 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:28.376754 13720 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:05:28.359373 13685 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:05:28.389370 13371 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:28.442708  9235 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:28.453979  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:28.445363 13659 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:28.455688 13601 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:05:28.457479  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 267418us
I20260709 13:05:28.458699  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.472755  9270 raft_consensus.cc:2368] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:05:28.459884 13788 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:28.383348 13799 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:05:28.460134  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.445623 13797 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.417936 13684 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:28.462213  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.512274  9267 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:28.514470  9120 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:28.463413  9238 leader_election.cc:400] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
I20260709 13:05:28.435602 13771 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:28.460157  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.467815  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.518096  9263 raft_consensus.cc:2410] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 13:05:28.469358  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.445993 13720 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:28.484720  9192 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:05:28.547971  9192 pending_rounds.cc:85] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 9
I20260709 13:05:28.548370  9192 pending_rounds.cc:107] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:05:28.485073  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.471925  9828 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:28.490492  9268 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:28.461033  9266 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:28.492100  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.503461  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.504801 13799 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:28.487968 13601 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:28.505668 13371 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:28.515717  9238 leader_election.cc:403] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:05:28.517359  9267 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:28.551270  9270 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:28.548700  9192 raft_consensus.cc:2889] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:28.558741  9193 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:28.561136  9269 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:28.561554  9271 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:05:28.567111  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 95295.4us
W20260709 13:05:28.568871  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 147973us
I20260709 13:05:28.517153 13538 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:28.452189 13683 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:28.548650 13804 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.485986 13478 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.551980 13720 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:05:28.552371 13232 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:28.551775 13601 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:28.551713 13799 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:28.572875  9826 leader_election.cc:336] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.500s (SENT)
I20260709 13:05:28.621644  9826 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:28.570714 13798 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:05:28.578328  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 141011us
I20260709 13:05:28.570184 13371 raft_consensus.cc:697] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:28.632617 13371 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.583231 13684 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:28.579198  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 159560us
I20260709 13:05:28.636137 13798 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:28.588855  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 227706us
I20260709 13:05:28.604555 13805 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:28.608942  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:28.625597 13827 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
W20260709 13:05:28.623409  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 163702us
I20260709 13:05:28.610064 13538 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.637668  9164 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:28.640482  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.640497  9191 raft_consensus.cc:2393] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:28.637168 13798 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.649106  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:28.656632  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 47768.8us
I20260709 13:05:28.657994  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:28.650579 13538 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.665653 13803 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:28.666229 13538 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:28.674919 13810 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:28.666245 13803 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:28.681753 13803 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.683144 13803 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:28.678638 13257 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:05:28.639076 13478 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:28.614210 13804 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:28.642848 13674 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:28.659324 13788 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:28.689488  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:28.949482  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 291586us
W20260709 13:05:28.757004  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.827823  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:28.844667  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.792956  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:28.830623  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 257768us
W20260709 13:05:28.953050  9826 leader_election.cc:336] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.519s (SENT)
I20260709 13:05:28.953487  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:28.786301 13836 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
W20260709 13:05:28.829226  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.807657  9853 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:05:28.902472  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 283032us
W20260709 13:05:28.723860  9264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2699) took 1299 ms (client timeout 1511 ms). Trace:
I20260709 13:05:28.817538  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:28.736131  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:28.728207  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 77983.2us
I20260709 13:05:28.823133  9859 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:28.824581  9856 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:05:28.789609  9087 leader_election.cc:336] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.505s (SENT)
I20260709 13:05:28.951224  9089 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:28.748941  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:28.826753  9122 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:28.957721  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:28.734030 13466 consensus_meta.cc:287] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.000s
W20260709 13:05:28.959337  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47003.7us
W20260709 13:05:28.786402 13830 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:05:29.003126  9087 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:29.003896  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 52753.3us
W20260709 13:05:28.904575 13624 consensus_meta.cc:287] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.627s	user 0.010s	sys 0.000s
W20260709 13:05:29.004000  9264 rpcz_store.cc:269] 0709 13:05:27.384560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:27.391012 (+  6452us) service_pool.cc:224] Handling call
0709 13:05:28.684073 (+1293061us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54001f8954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:28.684225 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968694656}
I20260709 13:05:29.002866  9858 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:29.112677  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.196200  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:28.683889 13331 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:29.130347  9269 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:05:29.137984  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.684719 13747 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:29.129910  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:28.842237 13682 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.687553 13823 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:29.137148  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:29.248530  9270 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:28.713406 13802 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:28.860508 13683 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:29.142102  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:28.858589 13832 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.687358 13712 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:28.712814 13336 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:28.818233 13232 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.790908 13681 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:28.861521 13824 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:28.862871 13804 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:28.684072 13659 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.457s
W20260709 13:05:29.147359  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 194323us
I20260709 13:05:29.138479 13810 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:29.128157 13797 consensus_meta.cc:287] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.609s	user 0.010s	sys 0.000s
I20260709 13:05:29.147802 13814 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.147886  9116 raft_consensus.cc:2410] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:29.148059  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 867) took 1124 ms. Trace:
I20260709 13:05:28.735095 13799 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.148389 13466 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:29.149945 13601 consensus_meta.cc:287] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.511s	user 0.003s	sys 0.007s
I20260709 13:05:29.150887  9266 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:29.274727  9270 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:29.151901  9268 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:29.153522  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:28.956676 13674 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:28.655405 13793 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.258903 13336 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:29.183050  9272 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:05:29.184973 13720 consensus_meta.cc:287] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.000s
I20260709 13:05:28.846304 13362 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:29.064342  9828 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.189013  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:29.198173  9269 pending_rounds.cc:85] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 10
I20260709 13:05:29.145063 13795 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:05:29.197983 13331 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:29.313683  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 249445us
W20260709 13:05:29.261152  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 471596us
I20260709 13:05:29.269716  9858 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:29.331457  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.197588 13504 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:05:29.216202 13624 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.276180  9263 raft_consensus.cc:2393] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:29.272140  9191 rpcz_store.cc:276] 0709 13:05:28.021223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.073715 (+ 52492us) service_pool.cc:224] Handling call
0709 13:05:29.145025 (+1071310us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400421b54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:29.145163 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195096832}
I20260709 13:05:29.248287 13807 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:05:29.258443 13712 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:29.260589 13232 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:29.258184 13824 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:29.246909 13156 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:05:29.250247 13682 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:29.294313 13597 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:29.261359 13747 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.278829  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:29.246135 13771 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.271085  9826 leader_election.cc:336] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.515s (SENT)
I20260709 13:05:29.333029 13331 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:29.332815  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 328795us
I20260709 13:05:29.356467 13674 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:29.349807  9855 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:29.181851  9267 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:05:29.386507  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 107754us
W20260709 13:05:29.348631  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 159727us
W20260709 13:05:29.313585  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 544448us
I20260709 13:05:29.261526 13681 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.293867 13336 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:29.358180 13803 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.001s
I20260709 13:05:29.352110  9271 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:05:29.356295 13849 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:05:29.361264  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:29.294219 13841 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 13:05:29.314071 13538 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.314386 13798 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:29.362082 13684 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:29.351135 13466 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:29.275450 13797 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.364169 13232 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:29.276901 13810 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.432296 13810 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:29.349220 13677 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:29.280471 13601 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.434211 13466 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:29.434840 13601 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:29.295413 13170 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:05:29.273303 13832 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:29.313385  9269 pending_rounds.cc:107] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:05:29.436954  9269 raft_consensus.cc:2889] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:29.437359  9269 pending_rounds.cc:107] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:05:29.437597  9269 raft_consensus.cc:2889] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:29.437915 13832 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:29.438977  9265 raft_consensus.cc:2410] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:05:29.368615 13824 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.439990 13824 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:29.332023 13362 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:29.384212 13771 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:29.445470 13681 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:29.370852 13846 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:05:29.447328 13771 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:29.448247  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 74984.3us
I20260709 13:05:29.448566  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:29.272264 13839 raft_consensus.cc:2764] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:05:29.410817  9860 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:29.411321  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:29.413866  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:29.415530  9115 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:29.420673  9118 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:29.421339  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:29.453596  9191 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:29.421924  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:29.423074 13478 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:29.423228 13538 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.423552  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.424029 13840 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:29.458840  9119 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:29.435407 13684 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:29.425504  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 154447us
W20260709 13:05:29.455950  9828 connection.cc:256] RPC call timeout handler was delayed by -1.0567494730000817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:05:29.314181 13720 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:29.429921 13797 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:29.445657 13362 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:29.369906 13828 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.421248 13471 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:05:29.479290  9118 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:29.452159  9115 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:29.385637 13747 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.427532 13835 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.277535 13814 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.467492 13822 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:29.471995  9123 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:29.420928 13852 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:05:29.467715 13674 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:29.478817 13788 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 13:05:29.455197  9859 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
W20260709 13:05:29.470430  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 59204.9us
I20260709 13:05:29.458490 13601 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:29.480187  9115 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:29.517992  9115 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:29.451278 13682 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:29.492826  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:29.521525  9860 raft_consensus.cc:2393] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:29.495486  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:29.498098  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:29.498693  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:29.501482 13538 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:29.504360 13624 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:29.506049 13804 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:29.508980 13683 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:29.510067 13793 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:29.510486 13674 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.463528 13771 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:05:29.470227 13478 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:29.472882 13840 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:29.543421 13601 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.580506 13840 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.581717 13601 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:29.493615  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:29.524202  9114 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:29.527447  9265 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:05:29.598264  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 104765us
I20260709 13:05:29.528179  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:29.599309  9122 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:29.529001  9191 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:29.535615  9267 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:05:29.540478  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.543134  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:29.600769  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.601199  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.601001  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:29.601642  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.604905  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:29.550249 13331 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.552023  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:29.553318  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:29.474794 13845 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:05:29.608666  9266 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:29.491051 13597 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.477190 13805 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 13:05:29.559265  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.559108  9087 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.759s (SENT)
I20260709 13:05:29.614687  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:29.560992  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:29.615868  9269 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:29.517241 13788 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:29.633977  9162 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:29.562466  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:29.525331 13812 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:29.562510  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 139090us
W20260709 13:05:29.568245  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 150542us
I20260709 13:05:29.642400  9161 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.570222 13712 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.511442 13466 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.584884 13747 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:29.588867 13814 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:29.525085 13809 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.590240  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:29.483505 13828 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.593739  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1326) took 1255 ms. Trace:
I20260709 13:05:29.677563  9855 rpcz_store.cc:276] 0709 13:05:28.338266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:28.348417 (+ 10151us) service_pool.cc:224] Handling call
0709 13:05:28.361529 (+ 13112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:29.385331 (+1023802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:29.593496 (+208165us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b5400506254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:29.593680 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2272382208}
I20260709 13:05:29.669982 13478 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:29.594620 13832 consensus_queue.cc:1059] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:29.595392 13684 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.595911  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:29.684476  9860 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:29.600921  9272 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:29.608626  9270 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:29.489123 13822 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:29.601814  9854 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:29.617667  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:29.624109  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 132615us
I20260709 13:05:29.609617 13793 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:05:29.631232  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 161936us
I20260709 13:05:29.689409  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:29.635553  9268 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:29.637382 13804 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:05:29.658020  9161 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.658160  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:29.657627  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:29.703444  9118 raft_consensus.cc:2368] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:05:29.660394  9122 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:29.663725  9116 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:29.661942  9162 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:29.563907 13817 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:05:29.640862 13771 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:29.594223 13674 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:29.606843 13807 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:05:29.642735 13841 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:29.659405 13156 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:29.542737 13795 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:29.663108 13504 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:29.656863  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:29.700335  9826 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.789s (SENT)
I20260709 13:05:29.742987  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:29.701653  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:29.658507 13852 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:05:29.707567  9825 leader_election.cc:400] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:05:29.744112  9825 leader_election.cc:403] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:05:29.678969 13803 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:29.708513  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 50567.1us
I20260709 13:05:29.656978 13682 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:05:29.673216 13336 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.660307 13812 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:29.725554  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63706.2us
I20260709 13:05:29.730576 13471 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:29.743235  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.675923 13809 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.750411  9191 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:29.660449 13170 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:05:29.724612 13771 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:29.767810  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.770629  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 83240.1us
I20260709 13:05:29.771713  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:29.693675 13822 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:29.730993 13681 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:29.742228 13156 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.773239 13471 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:29.782414  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 223671us
W20260709 13:05:29.801482  9087 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.802s (SENT)
I20260709 13:05:29.709265 13257 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:05:29.782601  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:29.784240 13771 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:29.786013  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.785661 13835 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:29.785192  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:29.778162 13788 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:05:29.822031  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.798044 13156 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:29.824112 13538 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:29.824885 13538 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:29.825371  9235 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:29.782549  9162 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:29.826807 13832 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:29.826161 13538 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.788216 13659 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:29.827623 13832 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:29.827944 13659 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.828076 13832 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.779007 13720 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:29.828245  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:29.787999  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 87708.4us
I20260709 13:05:29.743711 13841 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:29.814750  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:29.807713 13471 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:29.829519  9826 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.927s (SENT)
I20260709 13:05:29.829952  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.830745 13841 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:29.831261  9826 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.033s (SENT)
I20260709 13:05:29.816267 13624 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.834277  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.831701  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.839597 13257 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:05:29.841974  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 59495.7us
I20260709 13:05:29.828366 13659 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:29.780741 13795 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:29.849028  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 101258us
I20260709 13:05:29.850412  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.864257  9115 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:05:29.864809  9115 pending_rounds.cc:85] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 5
I20260709 13:05:29.865085  9115 pending_rounds.cc:107] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:05:29.865444  9115 raft_consensus.cc:2889] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:29.867653  9860 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:29.869481  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:29.873139  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:29.874189  9115 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
W20260709 13:05:29.865152  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 157784us
W20260709 13:05:29.903633  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72389.9us
W20260709 13:05:29.904422  9826 leader_election.cc:336] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.055s (SENT)
I20260709 13:05:29.904839  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:29.906381  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 157633us
I20260709 13:05:29.828483 13832 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:29.911718  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:29.907773  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:29.912429  9193 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:29.869108 13793 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:29.847692 13624 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:29.920607  9115 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:29.837918 13331 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:29.842994 13362 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:29.848014 13681 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:29.925789  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:29.908679  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:29.926589  9192 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:29.926658  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:29.927246  9122 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:29.868979 13720 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:29.918473  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:29.919109  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:29.919317  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:29.909034 13839 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:29.917526  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:29.922546  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 90904.4us
I20260709 13:05:29.922955  9189 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:29.848348 13257 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:29.921904  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:29.917264  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:29.929085  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:29.942898  9115 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:29.909986 13841 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:29.947691  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:29.952275  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47886.8us
W20260709 13:05:29.953006  9826 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.108s (SENT)
I20260709 13:05:29.952855  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:29.918200 13677 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:29.962230 13336 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:29.962661 13257 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.964785 13823 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:29.965837 13823 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:29.926271 13681 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.963168 13336 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:29.978411  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:29.921881 13852 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:05:29.994094  9122 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:29.955662 13839 raft_consensus.cc:697] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:30.024701  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 71716.1us
W20260709 13:05:30.034132  9826 leader_election.cc:336] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.182s (SENT)
I20260709 13:05:30.034544  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:29.958572 13797 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.029021 13170 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:29.955936 13884 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:29.986392 13681 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:29.967276 13720 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:29.998044 13257 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:30.041181 13881 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:30.044471 13793 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.040314 13812 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:30.052371 13866 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:05:29.984889 13336 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:29.988055 13876 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:29.985111 13677 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:30.088799  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 136053us
I20260709 13:05:30.037530 13828 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.038002 13170 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:30.051626 13874 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:05:30.089545  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 170314us
I20260709 13:05:30.039793 13684 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:30.059191  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 141744us
I20260709 13:05:30.092471  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:30.098299  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:30.108217  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:30.109185  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.114197 13878 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:05:30.115284 13872 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:05:30.093961 13170 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:30.131219  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 97117.8us
I20260709 13:05:30.133279  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:30.111233 13876 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:30.135063 13876 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:30.135565 13659 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.136117 13876 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.142634 13677 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.147516  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:30.148414  9122 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:30.088429 13336 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:30.067564 13795 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.043196 13797 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:30.107152 13881 raft_consensus.cc:3055] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:30.104692 13812 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:30.144321 13720 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.112285 13809 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:30.184473 13883 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:05:30.190972  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.116868 13832 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.180349  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:30.106766 13684 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.111203 13471 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.174794 13881 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:30.198824  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:30.198735  9826 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.295s (SENT)
I20260709 13:05:30.202745  9122 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:30.205503  9197 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 5.
I20260709 13:05:30.207865  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:30.219907 13895 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:30.219805 13720 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:05:30.221170 13892 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:30.221603 13840 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:30.222260 13803 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:30.223182 13881 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.204186 13659 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.225874 13797 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.226363 13684 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.176812 13839 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.344769 13659 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.317919 13841 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:30.345268 13659 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:30.229511  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.232338 13466 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:30.347503 13835 consensus_meta.cc:287] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260709 13:05:30.347541 13659 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.348218 13835 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10.
I20260709 13:05:30.348114 13466 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.349380 13805 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:30.350121 13466 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.233264  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:30.234025  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.331554 13683 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.351178  9856 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:30.351102 13683 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:30.234323 13871 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.428s	user 0.002s	sys 0.000s
W20260709 13:05:30.239356  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 151694us
I20260709 13:05:30.353968 13683 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:30.240348 13864 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.441s	user 0.001s	sys 0.000s
I20260709 13:05:30.354724 13894 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.240270 13597 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.355279 13894 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.355414 13597 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.357514 13894 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.357563 13597 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:30.243490 13875 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:05:30.244843  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.245544  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.251812 13686 raft_consensus.cc:3055] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:30.204615 13677 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:30.211953 13802 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:30.333777 13478 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.253134  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.368878  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:30.369377  9120 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:30.253203 13471 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.254634  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:30.256285  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.256755 13156 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:30.371475  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:30.372606  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 118084us
I20260709 13:05:30.259102  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:30.259850  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:30.373924  9190 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
W20260709 13:05:30.264047 13886 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:05:30.266952  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:30.374886  9114 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:30.374445 13681 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.375233 13821 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:05:30.375599  9826 leader_election.cc:336] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.818s (SENT)
I20260709 13:05:30.375787 13821 raft_consensus.cc:697] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:30.334225 13798 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:30.376768 13798 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.377169  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.376832 13821 consensus_queue.cc:237] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.379002 13798 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:30.366670 13822 consensus_meta.cc:287] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.001s
I20260709 13:05:30.380120 13822 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.269636  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:30.272677 13879 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.363s	user 0.002s	sys 0.009s
I20260709 13:05:30.382786  9197 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:30.323887  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:30.374908  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.383980  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:30.384574  9195 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:30.385361  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:30.341814 13793 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:30.331904  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.201352 13771 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.338747  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:30.339378  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:30.342825  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.349967 13805 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.364660 13797 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.258416 13832 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:30.382282  9114 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:30.385887  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.386626  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.421348  9116 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:30.210786  9859 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:30.386940  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:30.391090  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 121555us
I20260709 13:05:30.424862 13681 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.425611  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:30.426200  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:30.426836 13898 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.427062  9195 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:30.382337 13822 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:30.394696  9826 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.968s (SENT)
I20260709 13:05:30.406926  9198 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:30.429908 13799 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:30.430919  9194 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:30.409922  9196 raft_consensus.cc:2368] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:30.409617  9120 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
I20260709 13:05:30.372771 13156 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.413977  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.414776  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.364012 13894 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:30.436651 13894 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.438834 13894 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.362919 13686 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:30.365247 13677 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:30.439932 13677 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:30.440052  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:30.440709  9857 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:30.429947  9826 leader_election.cc:336] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.060s (SENT)
I20260709 13:05:30.427316  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.443822  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:30.445341  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 58520.5us
I20260709 13:05:30.445726  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:30.446317  9855 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:30.397397 13803 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:30.448391  9853 raft_consensus.cc:2410] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:05:30.448616  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.448877  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.409955 13809 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:30.343691 13840 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:30.475695  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 62659.5us
I20260709 13:05:30.476282  9862 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:30.476733  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.446985 13797 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.340173 13892 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.485606  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.390925 13478 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.489240  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:30.489569  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 59638us
I20260709 13:05:30.497179  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:30.473047 13471 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.451670 13802 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:30.490168  9855 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:30.452322 13170 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.473497 13809 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:30.427408 13898 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.512115  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:30.515640  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:30.421963 13793 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.517407  9089 leader_election.cc:400] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:30.515019 13471 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.518909 13771 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.513948 13802 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.519984 13601 raft_consensus.cc:2764] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:05:30.545403  9192 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:30.545859  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:30.546689  9855 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:30.520543 13840 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.499912 13835 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:05:30.555104 13892 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.520768 13336 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.556463  9122 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:30.559356 13336 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.525312 13504 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:30.529131 13478 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.529307 13895 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:30.561964 13895 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:30.562489 13895 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:30.566836  9826 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.248s (SENT)
I20260709 13:05:30.509356 13854 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.568320 13854 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.570215 13881 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:30.567430  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:30.548061 13909 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:05:30.574793 13471 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.545578 13802 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:30.577557 13892 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:05:30.578409 13902 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
I20260709 13:05:30.516131 13809 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:30.530241 13898 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.531279  9164 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:30.584408  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 53199us
W20260709 13:05:30.585217  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 635755us
I20260709 13:05:30.532039  9189 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:30.587296 13362 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:30.533550  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:30.533847  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.534735  9161 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:30.597880  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 63219.9us
I20260709 13:05:30.598891  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:30.535672 13761 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.499428 13682 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.484503 13686 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.499517 13814 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:30.542083  9089 leader_election.cc:403] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:30.615036 13170 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:30.615186  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 97924.9us
I20260709 13:05:30.614102  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.615072  9271 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:05:30.523418 13805 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:30.626410  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:30.626852  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:30.559423  9855 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
W20260709 13:05:30.591210  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 204076us
I20260709 13:05:30.591871  9123 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:30.592311 13898 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:30.593326  9190 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:30.593905  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:30.652689  9268 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:05:30.588012 13362 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:30.601152  9161 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:30.608444  9120 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:30.609488  9857 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:30.611799  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:30.662165  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 75318.3us
I20260709 13:05:30.663048  9858 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:05:30.614090  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:30.663883  9263 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:05:30.613390 13471 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:05:30.618072  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:30.670490  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.620087  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.673756  9854 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:30.676002  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.616479  9853 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:30.654558  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:30.610214 13712 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:30.661777  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 62087.6us
I20260709 13:05:30.680564  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:30.614506  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.680984  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:30.681388  9856 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:30.681543  9271 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:30.666579  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:30.669031  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:30.683522  9266 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:30.678704  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:30.625406 13852 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:30.686844  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.690676  9854 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:30.691596  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:30.672756  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.693747  9122 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:30.691648  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:30.699900  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.700170  9123 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:30.705420  9263 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:30.670032 13823 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:05:30.680323 13504 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:30.617592 13761 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.750274  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.751169  9853 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:30.752120  9859 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
W20260709 13:05:30.716591  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 62144.8us
I20260709 13:05:30.753197  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.717978  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:30.725371  9856 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:30.728928  9861 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:05:30.731309  9238 connection.cc:256] RPC call timeout handler was delayed by -3.4617573840000659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 13:05:30.755698  9238 connection.cc:256] RPC call timeout handler was delayed by -3.2826033850000158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:05:30.756305  9238 connection.cc:256] RPC call timeout handler was delayed by -2.842692392999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30560000000000004s for a response
I20260709 13:05:30.737890  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:30.757169  9269 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:30.720288  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:30.633895 13677 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.627472 13682 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:30.762048  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:30.763583  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.712704  9235 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:30.695880 13821 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:30.778846  9855 raft_consensus.cc:2368] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13.
W20260709 13:05:30.780557  9238 connection.cc:256] RPC call timeout handler was delayed by -1.9204847690000406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 13:05:30.781086  9238 connection.cc:256] RPC call timeout handler was delayed by -1.9006847690000086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
I20260709 13:05:30.782933  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:30.783591  9267 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:30.708148 13795 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:30.708670 13824 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
W20260709 13:05:30.781433  9238 connection.cc:256] RPC call timeout handler was delayed by -1.8196847689999913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
W20260709 13:05:30.810407  9238 connection.cc:256] RPC call timeout handler was delayed by -1.799884769000073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 13:05:30.810794  9238 connection.cc:256] RPC call timeout handler was delayed by -1.6981847690000222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
W20260709 13:05:30.811088  9238 connection.cc:256] RPC call timeout handler was delayed by -1.6954847690000179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
W20260709 13:05:30.811393  9238 connection.cc:256] RPC call timeout handler was delayed by -1.5586847690000241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
W20260709 13:05:30.811686  9238 connection.cc:256] RPC call timeout handler was delayed by -1.4200847690000273s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2339s for a response
W20260709 13:05:30.812036  9238 connection.cc:256] RPC call timeout handler was delayed by -1.397584768999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2364s for a response
W20260709 13:05:30.812405  9238 connection.cc:256] RPC call timeout handler was delayed by -1.1716847689999668s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2615s for a response
I20260709 13:05:30.808290  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.736845 13876 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:30.813982  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 108533us
I20260709 13:05:30.817821  9117 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:30.808255  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:30.773679 13879 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:05:30.736788 13822 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:30.817833  9235 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:30.814584  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.736969 13471 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:30.816844  9861 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:30.819886  9825 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:30.814940  9119 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:30.749507 13898 raft_consensus.cc:697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:30.834270  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.724581 13854 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:30.781867 13821 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:30.714872 13601 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.108s
I20260709 13:05:30.839964  9164 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:30.850802  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 132939us
I20260709 13:05:30.851523  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:30.846761 13156 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:30.855310  9267 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
W20260709 13:05:30.870296  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.873605  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.877372  9190 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:30.867650  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:30.878453 13331 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:30.784806 13877 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:05:30.878976  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:30.817301 13852 raft_consensus.cc:697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:30.880472  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:30.914626  9265 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:30.914780  9269 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:30.823918 13684 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:30.883214  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:30.866320  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:30.850484 13876 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:30.897572  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:30.898895  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:30.979342  9196 raft_consensus.cc:2410] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:05:30.845150 13538 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:30.985275  9116 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:30.899111  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:30.900017  9120 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:30.842700 13823 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:30.843842 13821 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:30.912807  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:30.918125  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.005575  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:30.878842 13799 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:05:30.949805  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:30.918628 13331 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:30.879767 13681 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:30.964071  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:30.896786 13674 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:30.964166  9853 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:30.968693  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:30.972009 13897 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:05:30.723639 13817 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:31.003413 13922 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:30.973994  9888 heartbeater.cc:646] Failed to heartbeat to 127.7.213.126:45141 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.7.213.126:45141 timed out after 15.000s (SENT)
I20260709 13:05:30.976217 13471 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:30.983570  9268 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:30.999402  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:30.989194 13823 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.010717  9192 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:31.011371  9194 raft_consensus.cc:2393] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 5: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:30.873486 13747 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:05:31.052307 13866 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:05:31.016734  9269 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
W20260709 13:05:30.921265  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.020991  9271 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:31.097085  9120 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:31.048424  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:31.048791  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.098373  9195 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:31.049683  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:31.004256 13911 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:05:31.060377  9861 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:31.075956  9856 raft_consensus.cc:2410] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:05:31.002626  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:31.102244  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:30.987563 13682 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:31.045403 13538 raft_consensus.cc:697] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:30.988018 13803 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:31.008194  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:31.086249  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 76223.6us
W20260709 13:05:31.087826 13924 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 13:05:31.088753 13918 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:05:31.023798  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:31.139891  9265 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:31.056190 13922 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.108160  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:30.978834 13684 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.076691  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.117944  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.059602 13674 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:31.003643 13884 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.083s
W20260709 13:05:31.135903  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 261169us
I20260709 13:05:31.083178 13832 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:31.135636  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.085887 13823 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:31.137754  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.121798 13890 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:31.164137  9192 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:31.137017  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:31.165218  9116 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:31.051060 13681 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.141098  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.190333  9196 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:31.082046 13797 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:31.084800 13747 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.143769  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:31.196303  9197 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:31.046634 13898 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.138383  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.167444  9195 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:05:31.198294  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 60035.6us
I20260709 13:05:31.198550 13683 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:31.199359 13683 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:31.199900 13915 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.200433 13915 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.202844 13915 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:31.203385 13896 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:31.204288  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:31.136480 13876 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.206424 13686 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:31.196588 13890 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.199525  9162 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:31.191422  9198 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:31.116161 13840 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:05:31.153147 13674 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:31.219937 13913 log.cc:927] Time spent T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:05:31.223210 13684 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:31.104111 13852 consensus_queue.cc:237] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:31.213992 13747 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:31.140182 13803 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:31.234637 13803 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:31.192971 13681 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:31.240588 13881 consensus_meta.cc:287] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.000s
W20260709 13:05:31.234813 13925 log.cc:927] Time spent T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 13:05:31.256214  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56778.6us
W20260709 13:05:31.257964  9266 consensus_meta.cc:287] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.001s
I20260709 13:05:31.258356  9266 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:31.279573  9162 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:31.242100 13895 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.001s
W20260709 13:05:31.339263  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56761.9us
W20260709 13:05:31.252890 13802 consensus_meta.cc:287] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.000s
W20260709 13:05:31.341331  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 125630us
I20260709 13:05:31.351933 13822 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:31.245436 13683 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:31.361246  9162 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:31.369204  9197 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 8.
W20260709 13:05:31.357182 13712 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.670s	user 0.005s	sys 0.005s
I20260709 13:05:31.156587 13331 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:31.375252  9854 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:31.398757 13674 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.376135 13814 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.357517 13895 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.377296 13684 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:31.362376 13538 consensus_queue.cc:237] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:31.410220 13936 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:31.424456 13936 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.428965  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:31.431952 13362 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:31.432760 13362 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.436652  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:31.438918  9087 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.588s (SENT)
I20260709 13:05:31.439836 13803 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.440352 13803 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.440750 13681 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:31.442855 13681 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:31.445308 13170 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:31.429625  9118 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:31.439390  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:31.452184  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:31.458796  9121 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:31.370362 13802 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:31.463778  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:31.447299 13821 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:31.460287  9087 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.733s (SENT)
I20260709 13:05:31.430723 13684 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.472892  9087 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:31.464455  9857 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:05:31.465853  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.413910 13793 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:31.454842 13336 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:31.458365 13682 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:31.477401 13809 consensus_meta.cc:287] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.892s	user 0.011s	sys 0.000s
I20260709 13:05:31.475734 13684 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:31.448666 13814 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.496445  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:31.484447 13845 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:31.502213  9119 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:31.507339  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:31.484565 13682 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:31.508112  9115 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:31.509325  9118 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:31.488520 13362 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:31.522198  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:31.544600  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:31.493775 13915 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:05:31.491842 13809 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.517403 13682 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:31.569746 13890 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:31.579175 13943 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:31.579767 13943 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:31.580801 13939 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:05:31.508289 13937 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:05:31.546857 13538 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:05:31.557422 13941 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:05:31.550374 13362 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:31.596971  9858 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:31.597915 13362 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:31.601218 13921 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:31.601577 13362 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:31.601687 13921 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:05:31.602845 13948 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.067s	user 0.000s	sys 0.003s
W20260709 13:05:31.605593 13942 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:05:31.613261 13943 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:31.620141 13922 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:31.621160  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:31.622118  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 77609.5us
I20260709 13:05:31.556768 13823 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.620205  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.627979  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.628767  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:31.629361  9266 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:31.632452  9198 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:31.626748  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.608587 13936 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:31.640058 13156 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.640499 13156 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:31.642153  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:31.652320 13823 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:31.648257 13821 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:31.649508 13852 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:31.680613  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:31.682240  9121 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:31.674531  9117 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:05:31.675576  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 47735.9us
I20260709 13:05:31.685591  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:31.686465  9862 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:31.701627  9266 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:31.644680 13934 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:31.705714  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 59959.3us
I20260709 13:05:31.713251  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:31.675403 13852 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:31.750669  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.706718 13685 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:31.757863  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.671566 13682 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.768668  9121 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:31.768985  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:31.772274  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:31.773649 13170 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:31.774288 13170 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:31.775415 13170 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:31.779942  9117 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:05:31.751924 13823 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:31.787988  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 163460us
I20260709 13:05:31.765765 13832 raft_consensus.cc:2764] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:31.755447 13931 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:31.789966  9087 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.652s (SENT)
W20260709 13:05:31.758927 13799 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.001s	sys 0.012s
I20260709 13:05:31.710628 13747 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.706724 13684 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.747007 13822 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:05:31.796528  9238 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.770614 13934 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:31.795887 13823 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.826475 13823 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:31.832016 13852 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:31.798596 13809 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:31.792371 13817 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:31.839856 13817 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:31.840286 13817 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:31.842546 13682 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:31.799371 13799 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.001s	sys 0.013s
W20260709 13:05:31.855434  9826 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.657s (SENT)
I20260709 13:05:31.855934  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:31.803285  9087 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.683s (SENT)
I20260709 13:05:31.856906  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:31.858546 13824 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:31.859511  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1138) took 1181 ms. Trace:
I20260709 13:05:31.859920  9854 rpcz_store.cc:276] 0709 13:05:30.678068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.699654 (+ 21586us) service_pool.cc:224] Handling call
0709 13:05:31.859351 (+1159697us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b5400524254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:31.859454 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1551631744}
I20260709 13:05:31.862694  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1308) took 1202 ms. Trace:
I20260709 13:05:31.863071  9858 rpcz_store.cc:276] 0709 13:05:30.660377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.722126 (+ 61749us) service_pool.cc:224] Handling call
0709 13:05:30.722428 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.597454 (+875026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.862485 (+265031us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b5400511654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:31.862641 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010862464}
I20260709 13:05:31.864305 13747 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:31.798116 13922 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:05:31.854383 13931 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:31.865957 13931 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:31.867803  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:31.870674 13677 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:31.873831  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 70587.1us
I20260709 13:05:31.872803  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2747) took 1255 ms. Trace:
W20260709 13:05:31.877893  9087 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.726s (SENT)
W20260709 13:05:31.879601  9087 leader_election.cc:336] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.774s (SENT)
I20260709 13:05:31.880075  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:31.883122  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:31.884815  9269 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:05:31.876226  9264 rpcz_store.cc:276] 0709 13:05:30.592270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.597376 (+  5106us) service_pool.cc:224] Handling call
0709 13:05:30.615355 (+ 17979us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:30.707706 (+ 92351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:30.707736 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:31.427969 (+720233us) spinlock_profiling.cc:243] Waited 8.75 ms on lock 0x7b2c00112948. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:05:31.447990 (+ 20021us) log.cc:844] Serialized 26 byte log entry
0709 13:05:31.700633 (+252643us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:31.700688 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.701934 (+  1246us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:31.846948 (+145014us) raft_consensus.cc:1685] finished
0709 13:05:31.846967 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.847160 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":54749,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":8580,"spinlock_wait_cycles":20129536,"thread_start_us":343471,"threads_started":2,"wal-append.queue_time_us":321722}
W20260709 13:05:31.885425  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 96411.4us
I20260709 13:05:31.893401  9238 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:31.893755  9087 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.930s (SENT)
I20260709 13:05:31.894927  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.895871 13835 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:31.896351 13835 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:31.897718 13597 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:31.876859 13832 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:31.888011 13915 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:31.898260 13832 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:31.898303 13677 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:31.898842 13915 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:31.899326 13832 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:31.899886 13915 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:31.900872 13832 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:31.901031  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1309) took 1170 ms. Trace:
I20260709 13:05:31.901544  9859 rpcz_store.cc:276] 0709 13:05:30.730664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.754111 (+ 23447us) service_pool.cc:224] Handling call
0709 13:05:30.754400 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:30.859630 (+105230us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:30.897413 (+ 37783us) log.cc:844] Serialized 14 byte log entry
0709 13:05:30.899109 (+  1696us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:30.899171 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:30.899214 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.655160 (+755946us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b48001bc258. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:05:31.900773 (+245613us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54007382d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:31.900828 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:31.900972 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81937,"prepare.queue_time_us":42202,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":787,"raft.queue_time_us":158,"raft.run_cpu_time_us":1824,"raft.run_wall_time_us":754948,"spinlock_wait_cycles":252962944}
I20260709 13:05:31.889115 13812 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:31.905999  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.888715 13931 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:31.897303 13835 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:31.906617 13812 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:31.910212  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.910377  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.928716  9193 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:31.930375  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:31.935050  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.938616  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:31.940145  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:31.934844  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:31.952963  9198 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:05:31.965045  9826 leader_election.cc:336] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.771s (SENT)
I20260709 13:05:31.910883 13934 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:05:31.969431  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 75711.7us
I20260709 13:05:31.912389 13686 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:05:31.965590  9826 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:31.982074  9087 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.178s (SENT)
I20260709 13:05:31.982257  9264 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:31.940246 13931 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:31.984186 13362 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:31.968870 13955 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:05:31.952059 13890 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:05:31.992058  9087 leader_election.cc:336] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.287s (SENT)
I20260709 13:05:31.992535  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:31.971441 13835 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:31.987529  9272 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:31.915820 13795 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:05:31.988796  9271 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:31.998756  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1150) took 1031 ms. Trace:
I20260709 13:05:31.969420 13922 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:31.993799  9265 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:31.994797  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.010368  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:32.010663  9268 raft_consensus.cc:2368] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11.
I20260709 13:05:32.010993  9854 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:31.936167 13681 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:05:32.012540  9859 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:31.996080  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:31.996398  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:31.936848 13812 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.015175  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.015411  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:31.997378  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:31.985531 13943 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:31.999289  9861 rpcz_store.cc:276] 0709 13:05:30.967848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.967990 (+   142us) service_pool.cc:224] Handling call
0709 13:05:31.051375 (+ 83385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:31.101665 (+ 50290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.998498 (+896833us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b54005ee550. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:31.998684 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190965504}
I20260709 13:05:32.018337  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.013847  9267 raft_consensus.cc:2376] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14. Re-sending same reply.
I20260709 13:05:32.014057  9270 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:32.019694  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:32.014317  9087 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.841s (SENT)
I20260709 13:05:32.014585  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:32.020884  9859 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:32.016809  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:05:32.021354  9087 leader_election.cc:336] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.868s (SENT)
I20260709 13:05:32.024727  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.025336  9263 raft_consensus.cc:2376] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11. Re-sending same reply.
I20260709 13:05:32.026178  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.026836  9263 raft_consensus.cc:2368] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 8. Current term is 9.
I20260709 13:05:32.028863  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:31.998502 13898 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:05:32.029700  9267 raft_consensus.cc:2393] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:32.030526  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.034255  9271 raft_consensus.cc:2368] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10.
I20260709 13:05:32.036964  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:32.039696  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:32.041210 13804 raft_consensus.cc:3055] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:32.041615 13804 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:32.041776 13362 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.042471 13804 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:32.044565 13935 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.663s	user 0.005s	sys 0.000s
I20260709 13:05:32.043640 13804 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:32.047600 13930 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.707s	user 0.001s	sys 0.002s
I20260709 13:05:32.027541  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.049476  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.049944  9263 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:31.992084 13890 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:32.053488 13890 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:32.054028  9270 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:32.054781 13601 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:32.054996  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.056206 13601 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:32.057626 13601 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:32.057986 13601 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:32.058845 13601 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:32.060052  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.061228  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.061556 13685 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:32.061720  9271 raft_consensus.cc:2376] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7. Re-sending same reply.
I20260709 13:05:32.062131 13685 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:32.062649  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.064503 13961 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:32.013157 13799 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:32.065659  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.066244  9264 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:32.066365 13674 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.067217 13812 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:32.068332  9265 raft_consensus.cc:2393] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:32.050966  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.069352  9268 raft_consensus.cc:2368] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
W20260709 13:05:32.029040  9189 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2233) took 1160 ms (client timeout 1531 ms). Trace:
I20260709 13:05:32.071497  9271 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:32.031538  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:32.070225  9189 rpcz_store.cc:269] 0709 13:05:30.868779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.868958 (+   179us) service_pool.cc:224] Handling call
0709 13:05:32.028845 (+1159887us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54000a6754. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:32.028987 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847852928}
I20260709 13:05:32.077121  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 887) took 1487 ms. Trace:
I20260709 13:05:32.078275  9191 rpcz_store.cc:276] 0709 13:05:30.589260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:30.609463 (+ 20203us) service_pool.cc:224] Handling call
0709 13:05:30.650018 (+ 40555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:30.650484 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:30.650524 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:30.683059 (+ 32535us) log.cc:844] Serialized 26 byte log entry
0709 13:05:31.032658 (+349599us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:31.032724 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:31.032829 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:32.047001 (+1014172us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:32.051729 (+  4728us) raft_consensus.cc:1685] finished
0709 13:05:32.051767 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:32.077040 (+ 25273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99027,"raft.queue_time_us":115,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":14077,"spinlock_wait_cycles":37060992,"thread_start_us":283017,"threads_started":1,"wal-append.queue_time_us":629535}
W20260709 13:05:32.117153  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.120584  9859 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:32.077116 13362 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:32.149855  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.150966  9087 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.006s (SENT)
I20260709 13:05:32.151392  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:32.152859  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:32.151546  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 81738.3us
W20260709 13:05:32.157226  9087 leader_election.cc:336] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.014s (SENT)
I20260709 13:05:32.157585  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.160145  9267 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:05:32.161667 13932 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.776s	user 0.001s	sys 0.007s
I20260709 13:05:32.164649 13684 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.164829  9267 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:32.166869  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.167199  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:32.169196  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 47459.6us
I20260709 13:05:32.169459 13805 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.170047 13805 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.171294  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.172049  9268 raft_consensus.cc:2368] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:32.172232 13805 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.171948 13677 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:32.172730 13805 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.173180 13805 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:32.173628  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:32.174233  9859 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:32.175362  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:32.175606  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.175976  9856 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:32.176465  9268 raft_consensus.cc:2393] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 5: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:32.148617 13674 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.152076 13934 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:32.200794  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:32.196018  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:32.201370  9118 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:32.197778  9854 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:32.194877  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:32.198998  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.203725  9270 raft_consensus.cc:2376] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7. Re-sending same reply.
I20260709 13:05:32.148955 13257 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:32.204245  9859 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:05:32.205245  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.203339 13934 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.208017 13934 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:32.207518 13257 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.198195 13799 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:32.209187 13799 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.209815 13257 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.211997  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:32.212592  9860 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:32.213220  9263 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:32.217532 13934 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.219868 13934 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.226274  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:32.227347  9116 raft_consensus.cc:2410] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:32.228375 13805 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:32.168730 13922 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:32.229292 13922 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:32.230347 13922 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.231266  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:32.231829  9116 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:32.239256  9116 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:32.240563  9122 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:05:32.242030  9265 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:32.242295  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:32.168730 13601 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:32.243760  9115 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:32.244670  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.244844  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:32.206178 13877 raft_consensus.cc:2764] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:32.248939 13961 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:32.249891 13824 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:32.252843  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:32.257046  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.260953  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:32.262135 13803 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.202517 13674 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.330214  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.270762  9118 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:31.544607  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:32.276762  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:32.334332  9854 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:32.354544  9856 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:05:32.286556  9123 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:32.298043  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:32.381129  9122 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:32.316794  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 64022.6us
I20260709 13:05:32.326910  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:32.328424 13937 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.368855 13961 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.259390 13921 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.376466  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.378405 13674 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.285732  9267 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:32.393168  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.380373  9118 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:32.150512 13817 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.388559  9123 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:32.392489  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.393797  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.394953  9122 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:32.295333 13922 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:32.403777  9271 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 7: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:32.410544  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.411432  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.329193 13677 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:32.413617  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.415388  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.420302  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.417058 13156 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:32.423841  9123 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:32.423272  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:32.425710  9122 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:32.404063 13824 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.426556  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.398571 13937 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.430977 13961 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.432034  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.433807 13894 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:32.404862 13805 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:05:32.425051 13921 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:32.436893  9266 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:32.535234  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:32.443511  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:32.444908 13156 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:05:32.481603  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.537789  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.485484  9826 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.867s (SENT)
I20260709 13:05:32.538502  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:32.487149  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 470404us
W20260709 13:05:32.488104  9190 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.013s	sys 0.011s
W20260709 13:05:32.539400  9190 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.013s	sys 0.011s
I20260709 13:05:32.548050  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.548775  9264 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:32.549784  9270 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:32.550009  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.551108  9265 raft_consensus.cc:2410] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:32.555202 13156 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:32.555727 13156 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:32.556007  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:32.556203 13156 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:32.556680  9194 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
W20260709 13:05:32.555272  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 69824.5us
W20260709 13:05:32.559417  9826 leader_election.cc:336] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.940s (SENT)
I20260709 13:05:32.560010  9826 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:32.563668  9266 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:32.566249  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.567329  9268 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:32.633293 13156 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.565538 13895 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:32.601688 13890 consensus_meta.cc:287] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.001s
I20260709 13:05:32.565905 13840 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:32.656343  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.584892 13684 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:32.661144 13156 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.626281 13817 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:32.660396 13970 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:05:32.663770  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:32.692075  9269 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:05:32.665071 13840 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:32.694856 13840 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.695325 13840 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.696753 13795 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:32.669206  9856 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:05:32.697284 13795 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.698501  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:32.699247  9856 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:32.699383 13795 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:32.672655  9857 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:05:32.707962  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 308962us
I20260709 13:05:32.675999  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:32.708993  9860 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:32.676720  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:32.677572  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:32.680416  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:32.669823  9854 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:32.651343 13807 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:32.653847 13890 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:32.748406  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:32.704061  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:32.728489 13824 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.729779 13937 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.652400 13835 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:05:32.584558 13812 consensus_queue.cc:1059] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
W20260709 13:05:32.706693 13685 consensus_meta.cc:287] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.003s
I20260709 13:05:32.777150  9264 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:32.772593  9089 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:32.783963  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.746095 13684 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:32.770226 13601 raft_consensus.cc:2764] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:05:32.853011 13799 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:32.851933 13684 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:32.831306 13971 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.373s	user 0.001s	sys 0.002s
I20260709 13:05:32.846521  9853 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:32.787425  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:32.790762 13824 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:32.847350 13804 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:32.770618 13807 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:32.869983  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 82641.2us
I20260709 13:05:32.787098 13938 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:05:32.871452 13804 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:32.875385 13840 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:32.879113  9272 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:32.790540 13685 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:32.869556 13824 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:05:32.882500  9087 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.114s (SENT)
W20260709 13:05:32.902108  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.902634  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.886045  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:32.906209  9825 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:32.858455 13835 consensus_queue.cc:1243] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:32.892488  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:32.894636  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 359497us
W20260709 13:05:32.890522  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 447074us
I20260709 13:05:32.900475  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:32.916234  9859 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:32.917754  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:32.918623  9856 raft_consensus.cc:2368] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9.
I20260709 13:05:32.919052  9164 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:32.913367  9858 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 13 LEARNER]: Advancing to term 14
I20260709 13:05:32.918336 13156 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:32.904385 13824 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:32.911864  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:32.908986 13840 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:32.913916 13877 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:32.920724  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:32.923938  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:32.928725  9117 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:32.915709  9270 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:32.925714 13601 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:32.907682 13921 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:05:32.925166  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:32.890290 13977 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:32.927860  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:32.933456  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:32.938694  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.924456  9087 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:32.987003  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:32.966609  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:32.940797  9859 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:32.920081 13805 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:05:32.989864  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:32.977780  9826 leader_election.cc:336] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.949s (SENT)
W20260709 13:05:32.987715  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 44363.6us
W20260709 13:05:32.972643  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:32.957218  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:32.989104  9862 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:32.971781  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:32.938252  9858 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:32.946179  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:32.990731  9856 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:32.990906 13156 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.025794 13156 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.005226  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:32.990506  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 66140.2us
I20260709 13:05:33.007953  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:33.008174  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:32.962167  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:32.930806 13478 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:32.996337  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:33.029567  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:33.024222  9855 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.029816  9193 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:05:32.987110 13978 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:05:33.028469  9195 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:33.032730  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:32.938776  9122 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:33.029526  9197 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:32.938011  9115 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:33.031363  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:33.031829  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:32.938776 13890 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:32.938489 13922 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:33.021158  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:33.049985  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72251.7us
W20260709 13:05:33.050892  9826 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.490s (SENT)
I20260709 13:05:33.051630  9826 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:32.990095 13877 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:33.044456  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:32.938717 13817 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:33.056173  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:33.056914 13156 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.047363  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.022158  9189 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:33.031364  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:33.005304 13170 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.042918  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:33.066713  9196 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:33.059876  9858 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:32.987778 13915 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.063230  9192 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:33.063441  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:33.028782 13840 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:33.072204  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:33.076790  9855 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:32.990393 13232 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:05:33.087127 13232 consensus_queue.cc:1243] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:05:33.079640  9238 leader_election.cc:400] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13.
I20260709 13:05:33.088047  9238 leader_election.cc:403] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:33.089380  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2774) took 1207 ms. Trace:
I20260709 13:05:33.090528  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:33.046433 13938 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:33.101001  9264 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:33.089308  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.089962  9266 rpcz_store.cc:276] 0709 13:05:31.881632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:31.882791 (+  1159us) service_pool.cc:224] Handling call
0709 13:05:33.089185 (+1206394us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54004434d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:33.089313 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250330496}
I20260709 13:05:33.104242  9189 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:33.105312  9197 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:33.106606  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:33.114482  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:33.077462 13478 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:33.115338  9192 raft_consensus.cc:2368] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
I20260709 13:05:33.107846 13685 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:33.108114  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:33.108355 13920 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:33.060525 13832 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:33.110941  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:33.082461 13817 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:33.112579  9263 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:33.081983 13840 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.149715 13804 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.054813 13934 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:05:33.109901 13802 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.099823 13747 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.119038 13983 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:33.119797  9118 pending_rounds.cc:85] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 3
I20260709 13:05:33.265825  9118 pending_rounds.cc:107] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:05:33.266243  9118 raft_consensus.cc:2889] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:33.131958 13807 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.148029 13877 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.189206  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:33.188323 13156 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:33.273808 13156 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:33.082885 13170 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.105806 13921 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13.
I20260709 13:05:33.091846 13915 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:33.189769  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 74382.1us
I20260709 13:05:33.114550 13814 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.106216 13601 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:05:33.190526  9115 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:33.059206 13922 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.215230  9196 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
W20260709 13:05:33.292549  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 240860us
I20260709 13:05:33.292806  9194 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:05:33.293187  9196 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:33.217917  9858 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:33.247290  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:33.251300  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:33.272681  9195 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: [11.12-11.12]
W20260709 13:05:33.276397  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:33.296113  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44927.6us
W20260709 13:05:33.255594 13985 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.209s	user 0.000s	sys 0.004s
I20260709 13:05:33.244369 13840 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.246907  9193 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 6 to 7, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:33.291983  9197 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:05:33.254784 13804 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:05:33.298350  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 259092us
I20260709 13:05:33.301254  9121 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:05:33.301163  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 274025us
I20260709 13:05:33.267508 13683 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:05:33.287498 13822 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.320978  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:33.269060 13747 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:33.299082 13980 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:33.326468  9858 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:33.312567 13890 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:33.343988  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 293120us
I20260709 13:05:33.325753 13983 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:33.324469  9087 leader_election.cc:400] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:33.346390  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.316095 13817 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.318811 13331 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:05:33.322425 13232 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:05:33.320387 13804 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:33.370868  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 264364us
I20260709 13:05:33.371480  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.372061  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.376713  9826 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.391399  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.316268 13478 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.348462 13983 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.350167 13802 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:33.366341 13937 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:33.329588 13822 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.341261 13890 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.423046  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:33.418493  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 104382us
I20260709 13:05:33.415336 13920 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:33.366019 13835 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 13:05:33.409885  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 109726us
I20260709 13:05:33.410197 13921 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.410312 13685 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:33.431322  9162 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:33.413021 13659 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:33.432636  9089 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:33.436192  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:33.441589  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:33.442147  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 70742.9us
I20260709 13:05:33.416618 13983 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:33.446976  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:33.447814  9116 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:33.420876 13996 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:33.429602 13934 raft_consensus.cc:2764] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:05:33.456823  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:33.484565 13989 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:33.484478 13761 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:33.455864 13824 consensus_meta.cc:287] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.528s	user 0.005s	sys 0.005s
I20260709 13:05:33.450446 13894 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:05:33.479389 13987 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:05:33.480561 13920 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:33.496604  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:33.481518 13170 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.421386 13802 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.452297 13683 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:05:33.444716 13832 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:05:33.505893 13601 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:33.454149 13990 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:05:33.482694 13685 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:05:33.438282 13921 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.510800  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:33.450413 13659 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.514827  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:33.514948  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:33.495895 13996 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.516690  9118 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:33.525784  9115 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:33.502354 13981 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:33.524637  9190 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:33.511238 13823 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:33.508214 14000 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:33.510453 13894 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:33.504578 13761 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:05:33.567178  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:33.556124  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 179461us
I20260709 13:05:33.514511 13915 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:33.500460 13824 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.519199 13601 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:33.569526  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:33.553544 13331 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:33.570652 13998 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:05:33.518095 13992 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:33.572151 13804 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:33.572347 13991 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 13:05:33.573943  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 136580us
I20260709 13:05:33.575452  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:33.589705  9118 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:33.539700 13747 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.591904  9123 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:33.595716  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:33.553242 14002 consensus_queue.cc:1059] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:05:33.580291 13232 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:33.580088 13994 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.573079 13922 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.579030 13601 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:33.605556 13961 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:33.605186 13840 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.607932 13890 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.609288 13659 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.610370  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2781) took 1063 ms. Trace:
I20260709 13:05:33.610507 13996 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.579344 14018 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:33.616118  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 46708.2us
I20260709 13:05:33.622488  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:33.626399  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:33.645937  9826 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.594522 13898 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:33.593580 13894 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.639236  9270 rpcz_store.cc:276] 0709 13:05:32.546808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.550556 (+  3748us) service_pool.cc:224] Handling call
0709 13:05:32.550826 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:32.930894 (+380068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.610127 (+679233us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54002a3354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:33.610306 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838769920}
I20260709 13:05:33.601982 13232 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.652665  9115 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:33.654822  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 222273us
I20260709 13:05:33.655479  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:33.657135  9862 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:33.655506  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:33.630193 13835 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:33.619918 13961 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:33.660024  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.659304  9238 leader_election.cc:400] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9.
I20260709 13:05:33.635003 13822 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.667057  9238 leader_election.cc:403] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:05:33.670617  9087 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.594s (SENT) [suppressed 33 similar messages]
I20260709 13:05:33.634121 14020 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:33.652765 13823 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:33.632825 13994 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:33.660518 13232 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.685626 13835 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:33.685810 13915 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:33.685649 13920 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9.
I20260709 13:05:33.664641 14000 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:33.693106 13823 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.710340 14000 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.712191 13823 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:33.713121 14000 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.663710 14002 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:33.658475 13898 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.717646 13994 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:33.718717 14012 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:05:33.720098 13915 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:05:33.720736 13961 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.721915 13961 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.696444 13817 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:05:33.722718 13802 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.672581 13996 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:33.704591 13824 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.681589 13804 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:33.724521 13804 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.724908 13804 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.725630 13840 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:33.673739 14014 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:33.727939 14014 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.666841 14015 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:33.730408 14014 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:33.732520 13975 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.795s	user 0.003s	sys 0.000s
I20260709 13:05:33.653564 14018 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:33.735901 14018 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.741320 13921 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.741129 13804 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.742084 13812 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.742794 13812 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.745311  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2537) took 1539 ms. Trace:
I20260709 13:05:33.745864 13812 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.749264 13804 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.760655 13915 raft_consensus.cc:3055] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:33.761242 13915 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:33.762660 13943 raft_consensus.cc:3055] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:33.762274 13915 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.763211 13943 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:33.764551 13943 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.745790  9119 rpcz_store.cc:276] 0709 13:05:32.205846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.205962 (+   116us) service_pool.cc:224] Handling call
0709 13:05:32.206231 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:32.206523 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.206551 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:32.211067 (+  4516us) log.cc:844] Serialized 26 byte log entry
0709 13:05:32.708635 (+497568us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:32.708674 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:32.708738 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:33.731878 (+1023140us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:33.744989 (+ 13111us) raft_consensus.cc:1685] finished
0709 13:05:33.745012 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.745249 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":412,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":109440256,"thread_start_us":497178,"threads_started":1,"wal-append.queue_time_us":684127}
W20260709 13:05:33.770678  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 100083us
W20260709 13:05:33.801362  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 115065us
I20260709 13:05:33.765820 13943 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:33.803988  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:33.802723 14002 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:05:33.810634 14014 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.811170 14002 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:33.812065 14002 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:33.811262 14014 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.812414 14002 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:33.813294 14002 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.814316 14014 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.814633 14002 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:33.823293 13938 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:33.824055 13938 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:33.824345 14030 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:33.825886 13920 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:33.826432 13920 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:33.827612 13920 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:33.834779  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 160129us
I20260709 13:05:33.835981  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:33.836319  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 870) took 1585 ms. Trace:
I20260709 13:05:33.837446  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:33.838703  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.838826 13990 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:33.839378 13990 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:33.840408 13990 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:33.841431 13824 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:33.841995 13824 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:33.842854 13824 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.844990 13804 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:33.835669 14025 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.849375 14025 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.799968 13232 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:33.851734 14025 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:33.851837  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.880775 13983 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.881759  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.883279  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:33.884195 14010 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:33.859300 13920 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:33.885989 13915 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:33.888132 13994 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:33.888875 13994 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:33.864953 13257 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:05:33.891376 13854 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.836910  9114 rpcz_store.cc:276] 0709 13:05:32.250627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.250739 (+   112us) service_pool.cc:224] Handling call
0709 13:05:32.251015 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:32.384609 (+133594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:32.384656 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:32.395850 (+ 11194us) log.cc:844] Serialized 26 byte log entry
0709 13:05:32.622113 (+226263us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:32.622152 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:33.835780 (+1213628us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b540023b9d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:33.835854 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:33.835922 (+    68us) raft_consensus.cc:1685] finished
0709 13:05:33.835945 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:33.836241 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12638,"raft.queue_time_us":7484,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":9464,"spinlock_wait_cycles":290063744,"thread_start_us":190060,"threads_started":1,"wal-append.queue_time_us":400534}
W20260709 13:05:33.895473  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:33.896077  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.860363  9238 leader_election.cc:400] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
I20260709 13:05:33.893720  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:33.907680  9115 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:33.879917  9123 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:33.913122  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:33.914036  9856 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:33.916687  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:33.918408  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:33.921464  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:33.921618  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:33.922308  9191 raft_consensus.cc:2393] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:33.923084  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:33.923950  9196 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:33.924649  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:33.925467  9266 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
W20260709 13:05:33.926954  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:33.928325  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:33.929169  9266 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:33.929335  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:33.917968  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:33.930603  9859 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:33.929394  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:33.871685 13832 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:05:33.938516  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:33.931771  9854 raft_consensus.cc:2410] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:33.939970  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:33.940656  9197 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:33.940709  9856 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
W20260709 13:05:33.946501  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 258182us
I20260709 13:05:33.947474  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:33.949141  9115 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:33.947948  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:33.952065  9194 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:33.878510 13232 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:33.950886  9161 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:33.918941 13921 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.868851 13979 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:05:33.952453 14020 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:33.932482 13854 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:33.942054 14005 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:05:33.946986 13684 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:05:34.001410  9087 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.755s (SENT)
I20260709 13:05:34.005703  9862 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:33.966363 13921 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.002658 13232 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:05:33.978336 13992 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:34.038022 13331 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.038749 13331 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:34.038983 14017 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:34.039544 14017 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.044476  9197 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:34.012096 13979 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:34.068765  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:34.074985  9123 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:34.078435  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:34.079357  9120 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:34.084676  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:34.052659 13977 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.040s	user 0.020s	sys 0.028s
I20260709 13:05:34.045682 13983 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:34.058409 13980 raft_consensus.cc:2764] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:05:34.058166 13684 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:34.015650 13992 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:34.092163  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:34.102346  9115 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:33.980078 14020 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:34.115213  9272 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 1021) took 1722 ms (client timeout 1771 ms). Trace:
I20260709 13:05:34.114450  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:34.087378 13979 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:34.088043 13934 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:34.059296 13601 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:34.100102  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.096933 13977 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.089s	user 0.020s	sys 0.028s
I20260709 13:05:34.104462 13684 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:34.093327 14019 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:34.115612  9272 rpcz_store.cc:269] 0709 13:05:32.392804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.392953 (+   149us) service_pool.cc:224] Handling call
0709 13:05:34.115031 (+1722078us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54003039d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:34.115151 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1086189312}
W20260709 13:05:34.122721  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 220428us
I20260709 13:05:34.132202  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:34.232576  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 100459us
W20260709 13:05:34.135648  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 185804us
W20260709 13:05:34.190033 14037 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:05:34.125140 13898 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:34.188823 14034 log.cc:927] Time spent T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:05:34.125483 13934 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:34.132297  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 181645us
I20260709 13:05:34.259157  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:34.179100 13979 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:34.239663  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 288877us
I20260709 13:05:34.241503  9089 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:34.250437  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:05:34.191244 13998 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:05:34.253284 13156 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:34.268518  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:34.122824 14007 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:34.273598 13937 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.240585 13961 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:34.275728 13601 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:34.318327  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 91665.3us
I20260709 13:05:34.278119 13934 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:34.225971 13832 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:34.281982 13817 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:05:34.289762  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:34.131901 13977 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:34.300652  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59211.8us
I20260709 13:05:34.316543 13961 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.308916  9265 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:05:34.309885  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.240585 14019 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.327389 13854 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:34.331565  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.334969  9268 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:05:34.293905 13795 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:05:34.336061  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.333726  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.313225 13920 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.316624 13804 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:34.343977  9089 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:34.304955  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:34.371935  9272 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.325172 13894 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:34.373406  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:34.373147  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 71739.2us
I20260709 13:05:34.374256  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.389137  9271 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:05:34.390251  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:34.375526  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:34.396121  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.324126 13937 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:34.359705 13961 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:34.401449  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.397475  9265 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:34.409412  9115 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:34.411549  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:34.414770  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:34.397725 13817 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:05:34.383700 13922 raft_consensus.cc:3055] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:34.394297 13982 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:34.375590 13795 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:34.376276 13894 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.394904 13990 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:34.317162 14007 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.377105 13921 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:34.467121  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.416287 14020 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:34.500243  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 124773us
I20260709 13:05:34.344386 13832 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:34.468775  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.468946  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.470939  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:34.460829  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 210379us
I20260709 13:05:34.379842 14019 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:34.329450 13684 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:34.520002  9855 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:34.395587 13920 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:34.506908  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 96030.4us
I20260709 13:05:34.533078 13937 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:34.533672 13920 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:34.534677 13920 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:05:34.484407 14050 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 13:05:34.536357 13938 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.712s	user 0.015s	sys 0.003s
I20260709 13:05:34.536845 13938 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:34.483858 14051 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:05:34.479084 13331 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.496335 13977 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:34.483229 14040 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:05:34.469635 14059 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:34.482892 13795 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.500450 13982 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:34.482286 13894 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:34.502715 13992 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:34.539722  9272 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:34.541467  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:34.551716  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:34.551904  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46340.7us
W20260709 13:05:34.553701  9087 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.602s (SENT)
I20260709 13:05:34.560830  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 897) took 2114 ms. Trace:
I20260709 13:05:34.574414  9854 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:05:34.576915  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:34.577718  9118 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:34.578107  9854 raft_consensus.cc:1217] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEARNER]: Deduplicated request from leader. Original: 8.5->[8.6-14.9]   Dedup: 8.6->[10.7-14.9]
I20260709 13:05:34.579025  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:34.580286  9115 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:34.582295  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:34.540123  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1329) took 1469 ms. Trace:
I20260709 13:05:34.552520  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.585152  9856 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:34.563385  9271 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:34.584268  9854 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 6
I20260709 13:05:34.588632  9854 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:05:34.589069  9854 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:34.591125  9854 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:05:34.589852  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:34.587833  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:34.592609  9856 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:34.593551  9854 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:34.596324 14047 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:34.544364  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:34.462523 13922 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:34.597112  9860 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:34.598184  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.602931  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:34.603299  9267 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:34.548749 13795 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:34.595492  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.584303  9853 rpcz_store.cc:276] 0709 13:05:33.071164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.209770 (+138606us) service_pool.cc:224] Handling call
0709 13:05:33.268019 (+ 58249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:33.273534 (+  5515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:33.273563 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:34.019256 (+745693us) spinlock_profiling.cc:243] Waited 9.37 ms on lock 0x7b2c00047988. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:05:34.019790 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:05:34.097167 (+ 77377us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:34.097219 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.097323 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:34.539821 (+442498us) raft_consensus.cc:1685] finished
0709 13:05:34.539849 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.540062 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12613,"raft.queue_time_us":6017,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5694,"spinlock_wait_cycles":21550976,"thread_start_us":76939,"threads_started":1,"wal-append.queue_time_us":516117}
I20260709 13:05:34.575899  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:34.585152  9268 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:34.572861  9198 rpcz_store.cc:276] 0709 13:05:32.446502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:32.446652 (+   150us) service_pool.cc:224] Handling call
0709 13:05:32.533131 (+ 86479us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:32.580559 (+ 47428us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:33.012459 (+431900us) log.cc:844] Serialized 14 byte log entry
0709 13:05:33.750319 (+737860us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b18002b7b60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:05:34.126226 (+375907us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:34.170889 (+ 44663us) log.cc:844] Serialized 52 byte log entry
0709 13:05:34.171081 (+   192us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:34.171414 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 13:05:34.171657 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.200066 (+ 28409us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:34.560597 (+360531us) raft_consensus.cc:1685] finished
0709 13:05:34.560618 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:34.560773 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16793,"prepare.queue_time_us":442327,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":93,"raft.queue_time_us":56018,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":5907,"spinlock_wait_cycles":16459520,"thread_start_us":770677,"threads_started":2,"wal-append.queue_time_us":1263455}
I20260709 13:05:34.605428  9862 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:34.591845  9861 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
W20260709 13:05:34.601313  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 75534.8us
I20260709 13:05:34.547652 14059 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:34.611030  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.611436  9123 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:34.611780 14007 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:34.612190  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.602077 13990 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:34.612470  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:34.612784  9267 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:34.613590  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.482630 13471 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:34.613286 13990 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.614287 13471 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.611521 14059 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:34.616477 13471 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:34.617156 14059 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:34.614209  9267 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:34.617669 13992 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:34.618191  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:34.552605 13822 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:05:34.622303 13674 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:05:34.560232 13943 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.755s	user 0.010s	sys 0.002s
I20260709 13:05:34.622428 13961 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.553593 13982 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:34.624697  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:34.545909 13915 consensus_meta.cc:287] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.659s	user 0.006s	sys 0.005s
I20260709 13:05:34.627447  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:34.628631  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:34.483443 13478 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:34.629140  9235 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:34.640528  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.563458 13331 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:34.657047  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:34.655311  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:34.659356  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.660029  9265 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:34.659541  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:34.546527 14017 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:34.669150  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.667165  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:34.647580  9264 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:34.672838  9272 raft_consensus.cc:2393] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:34.674968  9087 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:34.551826 13232 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 13:05:34.679232  9263 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:34.682232  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.640848 13601 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:34.657378 13938 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:34.597039 14047 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.684597  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.664158 13804 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.669469 13996 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:34.718257  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:34.665196 14049 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:34.663909 13937 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:34.678012 13835 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:34.732080  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.685578 14017 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:34.662755 13478 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.677851 13920 consensus_queue.cc:1059] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:05:34.743252  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:34.735394 13938 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:34.686342 14020 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:05:34.731272 13996 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:34.660877 13922 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.734370 14049 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.741431 13835 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:34.751295  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:34.768774  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:34.771709  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.773449  9265 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:34.748908 13685 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:34.749825 14036 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.757136 13938 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:34.788084 13938 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.791508 13761 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:34.792004 13761 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:34.792454 13761 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.762164 13996 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:34.798818  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:34.741777 13937 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.812456  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:34.817281  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.780277 13835 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:34.766762 14049 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:34.783764 13920 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:34.837598  9267 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:05:34.782837 13961 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:34.869984  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:34.783345 13915 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:34.830487  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:34.783584 13685 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:34.893285  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:34.766669 13934 consensus_queue.cc:1059] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:05:34.873245  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:34.896500  9265 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
W20260709 13:05:34.898603  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 275450us
I20260709 13:05:34.836090 14046 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:34.931061  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:34.839145 13920 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:34.838968 14057 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:34.953648  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:34.797530 14036 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:34.982170  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:35.017765  9190 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:35.019018  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 351920us
W20260709 13:05:35.024366  9826 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.722s (SENT)
W20260709 13:05:35.028007  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.104928  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 430058us
I20260709 13:05:34.898193 13685 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:34.907399 13674 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:34.905036 13990 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:34.908010 14020 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:34.957113 13934 consensus_queue.cc:1243] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:05:35.122017  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.132274  9826 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:35.122211  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.123215  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:35.125104  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:34.991786 13920 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:35.137118  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:35.141197  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:35.121731 13814 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.127875  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:35.150008  9193 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:35.119000  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:34.981418 14044 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.136333  9265 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:35.120720 14019 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:34.971252 14046 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:35.125025 13854 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.180934  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.186015  9190 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
W20260709 13:05:35.127635  9161 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.009s	sys 0.048s
W20260709 13:05:35.187196  9161 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.009s	sys 0.048s
I20260709 13:05:35.188547  9161 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:35.142369 14017 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:35.192898  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:35.107816 14057 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.185674 14047 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.135905 14020 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.196547 13854 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:35.210150  9854 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.003s	sys 0.007s
W20260709 13:05:35.211400  9854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.003s	sys 0.007s
I20260709 13:05:35.192631 13478 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.198266 14017 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.208168  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:35.130237 13990 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:35.216646  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.199887 14048 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:35.207124  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.215713  9265 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:35.201694 13943 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:35.226841  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.227728  9193 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:35.181831 13814 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.186627 14019 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.209216 14007 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:35.242503  9269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.027s	sys 0.039s
W20260709 13:05:35.247958  9269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.028s	sys 0.040s
I20260709 13:05:35.218287 14017 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:35.186738 14044 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.249315  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:35.252058  9193 raft_consensus.cc:2376] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10. Re-sending same reply.
I20260709 13:05:35.200973 14075 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:05:35.226300 13943 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.247464 14074 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.263001  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:35.258685 13982 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.000s	sys 0.020s
I20260709 13:05:35.286844  9269 raft_consensus.cc:2393] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:35.305351 14020 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.307093 14020 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:35.259706 13937 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.289959 14074 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.292230 13977 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:35.309573 13977 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:35.310637 13977 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.313073 14046 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:35.314810 13990 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.316870 14019 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:35.281921 13915 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.318958 13854 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:35.318636 13915 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:35.319444  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.320734 14073 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:05:35.322139 14069 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:05:35.321516 13915 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.224802 13674 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.274206 13943 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:35.291234 13232 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:35.324427 13232 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.308609 14036 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:35.305537 13795 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.000s
I20260709 13:05:35.315647  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:35.328306  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:35.286674 13982 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.000s	sys 0.021s
W20260709 13:05:35.325588 14064 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.430s	user 0.001s	sys 0.000s
I20260709 13:05:35.332032  9269 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 9.
I20260709 13:05:35.335940  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2809) took 1008 ms. Trace:
W20260709 13:05:35.303637 13938 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.000s
I20260709 13:05:35.337285  9271 rpcz_store.cc:276] 0709 13:05:34.327690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.335114 (+1007424us) service_pool.cc:224] Handling call
0709 13:05:35.335390 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.335544 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.335580 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.335617 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:35.335646 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.335697 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.335889 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.310655 14074 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.339298 13977 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:35.307643 14020 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.394461  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:35.417244  9116 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:35.416991  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:35.421924  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 156813us
I20260709 13:05:35.425312  9828 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:35.439917  9828 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:35.456712  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:35.461161  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2574) took 1175 ms. Trace:
I20260709 13:05:35.364522 13990 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.345333 14076 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:35.463356  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 322197us
I20260709 13:05:35.395107 14074 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.344431 13938 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:35.450467 13795 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:35.403410 14078 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:35.514163 14078 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.517124 14078 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.493692 14105 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:35.517748 14105 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:35.518121 14105 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:35.520187 14086 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.520742 14086 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:35.522830 13894 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.969s	user 0.009s	sys 0.001s
W20260709 13:05:35.492898  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 292858us
I20260709 13:05:35.527467  9118 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:35.527806  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:35.528412 14071 log.cc:927] Time spent T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 13:05:35.528805  9192 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:35.531064  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:35.532044  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:35.532956  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:35.533595  9117 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:35.536952  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:35.537734  9119 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:35.528487  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:35.538637  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:35.540439  9118 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:35.461624  9121 rpcz_store.cc:276] 0709 13:05:34.286293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.286422 (+   129us) service_pool.cc:224] Handling call
0709 13:05:34.316782 (+ 30360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:34.317101 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.317130 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:34.324466 (+  7336us) log.cc:844] Serialized 26 byte log entry
0709 13:05:34.631960 (+307494us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:34.631996 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.632065 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:35.460974 (+828909us) raft_consensus.cc:1685] finished
0709 13:05:35.460993 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.461107 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":404,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":6270,"thread_start_us":267893,"threads_started":1,"wal-append.queue_time_us":425027}
I20260709 13:05:35.540268  9117 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:35.498477 13990 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:35.450018 14075 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.455286 13817 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:35.462803 13804 consensus_meta.cc:287] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.733s	user 0.008s	sys 0.004s
I20260709 13:05:35.547658  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:35.549415  9197 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:35.551506  9193 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:05:35.551699 14103 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:35.552716  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:35.556152 13832 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:35.558207  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:35.539716 14036 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.560375 14036 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:35.562448 14036 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:35.537170 13835 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.697s	user 0.008s	sys 0.003s
I20260709 13:05:35.574491  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2162) took 1193 ms. Trace:
I20260709 13:05:35.574956  9263 rpcz_store.cc:276] 0709 13:05:34.380824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.573653 (+1192829us) service_pool.cc:224] Handling call
0709 13:05:35.573965 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.574146 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.574170 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.574211 (+    41us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:35.574242 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.574292 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.574429 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.522856 14086 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:35.577975  9192 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:35.540251 14057 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.566303 13804 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:35.563702 13814 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:35.510005 14074 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:35.600804  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2163) took 1219 ms. Trace:
I20260709 13:05:35.605273  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 2151) took 1267 ms. Trace:
I20260709 13:05:35.523474 13894 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.608563  9267 rpcz_store.cc:276] 0709 13:05:34.338143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.617936 (+279793us) service_pool.cc:224] Handling call
0709 13:05:35.597861 (+979925us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54002a3d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:35.605189 (+  7328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502121216}
I20260709 13:05:35.568635 13854 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:35.603184  9265 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:35.609370  9263 rpcz_store.cc:276] 0709 13:05:34.381169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.579663 (+1198494us) service_pool.cc:224] Handling call
0709 13:05:35.600100 (+ 20437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.600374 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.600410 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.600466 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:35.600511 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.600592 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.600735 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.612306  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:35.617295  9189 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:35.618626  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.619295  9192 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:35.623157  9114 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:35.629879  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:35.630192  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2810) took 1164 ms. Trace:
I20260709 13:05:35.630683  9272 rpcz_store.cc:276] 0709 13:05:34.465712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.611517 (+1145805us) service_pool.cc:224] Handling call
0709 13:05:35.629623 (+ 18106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.629846 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.629872 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.629915 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:35.629947 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.630003 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.630136 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.631533 13938 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:35.617174 13854 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:35.630000 14044 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.634479 13854 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:35.565939 13817 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.638929  9115 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:35.639564  9198 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:35.640019  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.641157  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.579056 13835 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.651346  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.652240  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:35.698232  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.728220 13996 consensus_meta.cc:287] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.902s	user 0.008s	sys 0.002s
I20260709 13:05:35.728775 13996 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.617669 13659 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:35.735216  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.797852  9197 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:35.567251 14075 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.838490  9197 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:35.854394  9192 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:05:35.859655  9189 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
W20260709 13:05:35.860067  9826 leader_election.cc:336] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.906s (SENT)
W20260709 13:05:35.772089 13761 consensus_meta.cc:287] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.979s	user 0.009s	sys 0.004s
I20260709 13:05:35.853758  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.869318  9197 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:35.756385 14002 raft_consensus.cc:3055] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:35.861959  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.865047  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:35.865002  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 448107us
I20260709 13:05:35.741809 13832 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:35.879344  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:35.879977  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.821122 13938 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.871711  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2812) took 1405 ms. Trace:
I20260709 13:05:35.873657  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.643898 13943 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.875084  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:35.842427 13804 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:35.868091 14049 consensus_meta.cc:287] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.998s	user 0.009s	sys 0.002s
W20260709 13:05:35.861519  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.874632  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:35.889659  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:35.891983 14078 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:35.892539 14078 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.893018 14078 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:35.854138 14057 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.883852  9265 rpcz_store.cc:276] 0709 13:05:34.466328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.616352 (+1150024us) service_pool.cc:224] Handling call
0709 13:05:35.621341 (+  4989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.621865 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:35.771857 (+149992us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.871184 (+ 99327us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.871254 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:35.871319 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.871419 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.871645 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":149571,"prepare.run_wall_time_us":6710,"raft.queue_time_us":3208,"thread_start_us":98909,"threads_started":1}
I20260709 13:05:35.863605 14085 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:35.919657  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2813) took 1370 ms. Trace:
I20260709 13:05:35.921783  9265 rpcz_store.cc:276] 0709 13:05:34.549330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.918515 (+1369185us) service_pool.cc:224] Handling call
0709 13:05:35.918923 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.919174 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.919210 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.919294 (+    84us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:35.919340 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.919413 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.919579 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.840226 13659 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:35.872838 14002 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:35.930243  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2814) took 1327 ms. Trace:
I20260709 13:05:35.882741 13814 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:35.921036 13938 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:35.945863  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48183.3us
I20260709 13:05:35.949313 13938 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:35.930102  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 473423us
W20260709 13:05:35.942818  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52175.8us
I20260709 13:05:35.882308 13980 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:05:35.924084 14020 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.956400  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.950964 14002 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:35.956763  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.957409 14002 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:35.904295 14049 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:35.959157  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:35.960749 14049 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:35.904496 13257 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:35.961627 13257 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:35.962035 13257 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:35.964706  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2156) took 1585 ms. Trace:
I20260709 13:05:35.964799  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:35.952530 13817 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:35.873265 13761 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.888512 13832 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.939638  9265 rpcz_store.cc:276] 0709 13:05:34.602577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.923227 (+1320650us) service_pool.cc:224] Handling call
0709 13:05:35.923606 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.929329 (+  5723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.929375 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.929437 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:35.929484 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.929569 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.930172 (+   603us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.968640  9192 raft_consensus.cc:2410] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:05:35.930102 14085 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.979015  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2815) took 1372 ms. Trace:
I20260709 13:05:35.987048 13835 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:35.986099  9265 rpcz_store.cc:276] 0709 13:05:34.607019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.978268 (+1371249us) service_pool.cc:224] Handling call
0709 13:05:35.978540 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.978722 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.978747 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.978786 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:35.978822 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.978873 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.978963 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.904223 13804 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:35.960775 13920 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.811s	user 0.008s	sys 0.001s
I20260709 13:05:35.896293 14057 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:35.965173  9268 rpcz_store.cc:276] 0709 13:05:34.378884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.559317 (+1180433us) service_pool.cc:224] Handling call
0709 13:05:35.559619 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.564778 (+  5159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.564811 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:35.565260 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:05:35.745691 (+180431us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:35.745751 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.745828 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:35.964512 (+218684us) raft_consensus.cc:1685] finished
0709 13:05:35.964530 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.964655 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22123,"raft.queue_time_us":711,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":4766,"thread_start_us":144066,"threads_started":1,"wal-append.queue_time_us":343616}
I20260709 13:05:35.993446  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2165) took 1332 ms. Trace:
I20260709 13:05:35.993975  9268 rpcz_store.cc:276] 0709 13:05:34.662827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.992389 (+1329562us) service_pool.cc:224] Handling call
0709 13:05:35.992771 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.993016 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.993052 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.993120 (+    68us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:35.993167 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.993239 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:35.993376 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:35.967914 13478 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:35.966449  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:35.971179 13814 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:35.996236  9268 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:05:36.000106  9268 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 8
I20260709 13:05:35.978112 13990 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.000623  9268 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 14.9
W20260709 13:05:35.984769 13685 consensus_meta.cc:287] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.858s	user 0.004s	sys 0.008s
I20260709 13:05:36.003388  9268 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:36.016865 13659 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.017297 13804 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:36.024823 14082 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.514s	user 0.003s	sys 0.000s
I20260709 13:05:36.027199  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2154) took 1688 ms. Trace:
I20260709 13:05:35.996809 14105 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:35.999571 13478 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:36.027719  9271 rpcz_store.cc:276] 0709 13:05:34.338875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.343041 (+1004166us) service_pool.cc:224] Handling call
0709 13:05:35.343354 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.343719 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.343781 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:35.365195 (+ 21414us) log.cc:844] Serialized 26 byte log entry
0709 13:05:35.843607 (+478412us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:05:35.843672 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:35.843785 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.026907 (+183122us) raft_consensus.cc:1685] finished
0709 13:05:36.026935 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.027112 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":48135,"thread_start_us":360004,"threads_started":1,"wal-append.queue_time_us":450785}
I20260709 13:05:36.039769  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2817) took 1290 ms. Trace:
I20260709 13:05:36.040199  9271 rpcz_store.cc:276] 0709 13:05:34.749089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.038666 (+1289577us) service_pool.cc:224] Handling call
0709 13:05:36.039089 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.039358 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.039394 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:36.039449 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:36.039492 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.039560 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.039688 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:36.004346  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2168) took 1340 ms. Trace:
I20260709 13:05:36.042878  9268 rpcz_store.cc:276] 0709 13:05:34.665165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.995160 (+1329995us) service_pool.cc:224] Handling call
0709 13:05:35.995505 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.004098 (+  8593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.004293 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:36.034797 13478 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:36.049482 13257 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.051301  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2819) took 1255 ms. Trace:
I20260709 13:05:36.051666  9268 rpcz_store.cc:276] 0709 13:05:34.796090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.050527 (+1254437us) service_pool.cc:224] Handling call
0709 13:05:36.050821 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.050988 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.051011 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:36.051051 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:36.051095 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.051146 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.051250 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:36.062592 14085 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.004916 13814 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.033010 13943 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.065269 13814 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.035815 13685 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.068785 13685 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:36.035178 14017 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.763s	user 0.009s	sys 0.000s
I20260709 13:05:36.069558 14017 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:36.071033 13977 consensus_meta.cc:287] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.616s	user 0.002s	sys 0.006s
I20260709 13:05:36.071861  9122 raft_consensus.cc:2410] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:05:36.072813  9267 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:36.072942 13761 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.073972 13814 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:36.074576 13814 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.076560 13232 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:36.081903 14098 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.589s	user 0.001s	sys 0.001s
W20260709 13:05:36.034135 14079 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.553s	user 0.000s	sys 0.002s
W20260709 13:05:36.036032 14095 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:05:36.098773 13257 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.034210 14074 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.028692 14075 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.001245 13920 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.096939 13996 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.031504 14105 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:36.111686 14017 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.117865 13257 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.121039 13814 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.121775 13934 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.122803 13478 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:36.187232  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.189368  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.190451  9856 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:36.130537  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.135411  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:36.199731  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.136591  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.137193  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.137338  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.139290 13684 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:36.146836  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.207861  9859 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:36.153906  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:36.155241  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:36.158551  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.159817  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:36.170954  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.171144  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.211612  9862 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
W20260709 13:05:36.211954  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52251us
I20260709 13:05:36.211642  9860 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:36.171321  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.213874  9854 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:05:36.172226  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.174021 13257 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.215078  9861 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:36.175427  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.216293  9855 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:36.136281 14075 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:36.206372  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.220712  9853 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:36.200860  9195 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
W20260709 13:05:36.218422  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 175723us
I20260709 13:05:36.222589  9117 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:36.227316  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:36.227258  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45673.6us
I20260709 13:05:36.229108  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.230455  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:36.232326  9089 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:36.233431  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:36.235167  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.237059 14024 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:36.236943 14078 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:36.237560 14024 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.237948 14024 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.228384 14117 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:36.239869 14017 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.240490 14017 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.240959 13601 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:36.241627 13761 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:36.242211 13761 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:36.242688 14017 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:36.244719  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 89579us
W20260709 13:05:36.245075 14112 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:05:36.245838  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:36.246260 14057 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:36.246647 13996 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:36.247054 13996 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:36.247387 13996 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:36.218043 13934 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:36.232179 13921 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:36.292613  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 46828.1us
W20260709 13:05:36.304114  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 356717us
W20260709 13:05:36.293195  9828 leader_election.cc:336] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.540s (SENT)
I20260709 13:05:36.298655  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.298902  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.239466 14078 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.302485  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:36.306567  9197 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:36.306505  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 62749.3us
I20260709 13:05:36.307006  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:36.296075  9264 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 1038) took 1778 ms (client timeout 1540 ms). Trace:
I20260709 13:05:36.304749  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:36.307667  9264 rpcz_store.cc:269] 0709 13:05:34.517493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.541208 (+ 23715us) service_pool.cc:224] Handling call
0709 13:05:36.251762 (+1710554us) spinlock_profiling.cc:243] Waited 63.7 ms on lock 0x7b5400302fd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:36.295975 (+ 44213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146621824}
I20260709 13:05:36.308728  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:36.310144  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:36.303972 14020 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:36.312130 14020 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.312700 14015 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:36.313051 14015 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:36.314178 14020 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.315440 14017 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:36.299630  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.303241  9117 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:36.313853 14015 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:36.318415  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1054) took 1296 ms. Trace:
I20260709 13:05:36.323889  9264 rpcz_store.cc:276] 0709 13:05:35.022539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.311118 (+1288579us) service_pool.cc:224] Handling call
0709 13:05:36.317809 (+  6691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.318016 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.318054 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:36.318110 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:36.318154 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.318223 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.318351 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:36.319427  9825 connection.cc:256] RPC call timeout handler was delayed by -1.146242261999987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
W20260709 13:05:36.329954  9825 connection.cc:256] RPC call timeout handler was delayed by -1.0834388799999033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
I20260709 13:05:36.310886  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:36.254803 13852 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:36.243177 13761 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.338922  9861 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:36.300865 14105 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.305286 13814 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:36.222311 13992 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:05:36.322281 13921 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.305133 13601 raft_consensus.cc:697] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:36.352644  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.323465 14017 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:36.255549 14075 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.354833  9855 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:36.353550  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.355031  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.370499  9854 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:36.349313 14015 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.370942  9853 raft_consensus.cc:2410] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:05:36.354473 14078 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:36.359148 13478 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.380807  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.356997  9861 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:36.374289  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.375718  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.348284 13814 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.384763  9855 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:36.384970  9860 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:36.353771 14105 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.389813  9197 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:36.354234 13674 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:05:36.407454  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.407788 13934 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:36.407950 13674 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:36.380347 14017 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.408783  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:36.410200  9825 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:36.411283  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:36.411991 13835 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:36.413087  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.391165  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.413712  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:36.394650  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.414819  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:36.415334  9118 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:36.417093  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2811) took 1951 ms. Trace:
I20260709 13:05:36.417462  9267 rpcz_store.cc:276] 0709 13:05:34.466063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.614314 (+1148251us) service_pool.cc:224] Handling call
0709 13:05:35.614687 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.073264 (+458577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.416872 (+343608us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54000faad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:36.417035 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1053133696}
I20260709 13:05:36.419440  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1041) took 1901 ms. Trace:
I20260709 13:05:36.394804  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:36.398952 13601 consensus_queue.cc:237] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:36.402082  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2816) took 1745 ms. Trace:
W20260709 13:05:36.426563  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.425457  9265 rpcz_store.cc:276] 0709 13:05:34.657001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.988707 (+1331706us) service_pool.cc:224] Handling call
0709 13:05:35.989051 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.989842 (+   791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.989878 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.043209 (+ 53331us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.207418 (+164209us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:36.207456 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.207518 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.401865 (+194347us) raft_consensus.cc:1685] finished
0709 13:05:36.401888 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.402024 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7603,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5833,"thread_start_us":158776,"threads_started":1,"wal-append.queue_time_us":351844}
I20260709 13:05:36.427125  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:36.378345 14078 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.405090 13921 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.407162  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.419795  9263 rpcz_store.cc:276] 0709 13:05:34.518116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.620698 (+1102582us) service_pool.cc:224] Handling call
0709 13:05:35.645487 (+ 24789us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.646001 (+   514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:35.646039 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:35.755822 (+109783us) log.cc:844] Serialized 26 byte log entry
0709 13:05:35.984411 (+228589us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:36.002494 (+ 18083us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.002999 (+   505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.003068 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.419256 (+416188us) raft_consensus.cc:1685] finished
0709 13:05:36.419278 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.419388 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13268,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":197,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":123450,"thread_start_us":106381,"threads_started":1,"wal-append.queue_time_us":242070}
I20260709 13:05:36.423072  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:36.430307 13943 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:36.437714  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:36.424052  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.438587  9861 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:36.438644 14014 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:36.411077 14024 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.439486 13835 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:36.441078 14024 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:36.385186 14052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.011s	sys 0.028s
W20260709 13:05:36.441591 14052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.011s	sys 0.028s
I20260709 13:05:36.434262  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.441896  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2803) took 2136 ms. Trace:
I20260709 13:05:36.442355  9855 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:36.442996 13835 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:05:36.444425 13854 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.809s	user 0.010s	sys 0.001s
I20260709 13:05:36.442252  9270 rpcz_store.cc:276] 0709 13:05:34.305217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.310495 (+  5278us) service_pool.cc:224] Handling call
0709 13:05:34.310873 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:34.311323 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:05:34.367527 (+ 56204us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.883663 (+1516136us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b1800406840. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:05:36.184309 (+300646us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.218146 (+ 33837us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.218296 (+   150us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:36.218334 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.218399 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.441606 (+223207us) raft_consensus.cc:1685] finished
0709 13:05:36.441695 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.441818 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14141,"prepare.queue_time_us":212047,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":57468,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":41364,"spinlock_wait_cycles":3345792,"thread_start_us":340427,"threads_started":2,"wal-append.queue_time_us":2061792}
I20260709 13:05:36.444835 13854 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:36.447178  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.447917  9859 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:36.449142 13854 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:36.451671 13683 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:36.454722 14116 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:36.410224 14017 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.456286 14020 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:36.481799  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.458048 14075 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.459599  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:36.489578  9193 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.462028  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:36.465389  9118 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:36.394353 13814 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:36.417261 13996 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.392236 13832 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:36.480767  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 277480us
I20260709 13:05:36.437155 14078 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:36.473101 13683 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:36.466868 13920 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.513124  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:05:36.516978  9020 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36626 (request call id 360) took 19272 ms (client timeout 15000 ms). Trace:
I20260709 13:05:36.520242  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:36.532969  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:36.536377  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:05:36.520251 14002 consensus_meta.cc:287] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.007s
W20260709 13:05:36.520697  9020 rpcz_store.cc:269] 0709 13:05:17.244983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:17.254053 (+  9070us) service_pool.cc:224] Handling call
0709 13:05:36.516884 (+19262831us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:28.628170 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:28.628895 (+   725us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:28.628951 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:05:28.628975 (+    24us) tablet.cc:662] Decoding operations
0709 13:05:28.631071 (+  2096us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:28.631187 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:28.631219 (+    32us) tablet.cc:685] Acquiring locks for 26 operations
0709 13:05:28.662534 (+ 31315us) tablet.cc:701] Row locks acquired
0709 13:05:28.662587 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:05:28.662767 (+   180us) write_op.cc:270] Start()
0709 13:05:28.662921 (+   154us) write_op.cc:276] Timestamp: P: 1783602328662708 usec, L: 0
0709 13:05:28.662959 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:05:28.663942 (+   983us) log.cc:844] Serialized 10682 byte log entry
0709 13:05:29.355377 (+691435us) op_driver.cc:464] REPLICATION: finished
0709 13:05:29.550182 (+194805us) write_op.cc:301] APPLY: starting
0709 13:05:29.550296 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:30.218447 (+668151us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:30.218574 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:30.538013 (+319439us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:30.538701 (+   688us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=0,mrs_lookups=0
0709 13:05:30.538788 (+    87us) write_op.cc:312] APPLY: finished
0709 13:05:30.539596 (+   808us) log.cc:844] Serialized 228 byte log entry
0709 13:05:30.779269 (+239673us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:30.779474 (+   205us) write_op.cc:454] Released schema lock
0709 13:05:30.779581 (+   107us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:30.846772 (+ 67191us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:30.849083 (+  2311us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2123695,"reported_tablets":46,"spinlock_wait_cycles":3328,"child_traces":[["op",{"apply.queue_time_us":194451,"apply.run_cpu_time_us":51195,"apply.run_wall_time_us":1265050,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112859,"num_ops":26,"prepare.queue_time_us":157446,"prepare.run_cpu_time_us":12201,"prepare.run_wall_time_us":44877,"raft.queue_time_us":467464,"raft.run_cpu_time_us":72616,"raft.run_wall_time_us":275863,"replication_time_us":669952,"thread_start_us":935169,"threads_started":5,"wal-append.queue_time_us":469637,"wal-append.run_cpu_time_us":48148,"wal-append.run_wall_time_us":2926095}]]}
I20260709 13:05:36.547088  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:36.516050 13683 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:36.490954 14020 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:36.557863  9123 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.541126 13920 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:36.562040  9193 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:36.546980 13981 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:36.567523  9263 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:36.561534 13920 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:36.556353 13921 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:36.555258 13761 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.583431  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:36.570796 14127 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:36.581766 13920 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:36.585839 13761 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.588979 14024 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:36.583117 13921 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:36.597054 14127 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:36.622933  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2552) took 1208 ms. Trace:
I20260709 13:05:36.616818 13921 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.632261  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43782 (request call id 2579) took 1102 ms. Trace:
I20260709 13:05:36.623554  9114 rpcz_store.cc:276] 0709 13:05:35.414667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.415548 (+   881us) service_pool.cc:224] Handling call
0709 13:05:35.415886 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.623698 (+207812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.622610 (+998912us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54001202d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:36.622854 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476230528}
I20260709 13:05:36.637730  9856 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEARNER]: Advancing to term 15
I20260709 13:05:36.640605  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.641894  9860 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 16: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:36.650146  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.651068  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.651418  9858 raft_consensus.cc:2410] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:05:36.632853  9122 rpcz_store.cc:276] 0709 13:05:35.525848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.531821 (+  5973us) service_pool.cc:224] Handling call
0709 13:05:36.620997 (+1089176us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400284e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:36.628204 (+  7207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239422848}
I20260709 13:05:36.651769  9861 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:36.651757  9123 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:36.623931 14024 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:36.666862  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2818) took 1879 ms. Trace:
W20260709 13:05:36.631417 14123 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:05:36.665181  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:36.674690  9122 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:36.666987  9857 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:36.587970 13683 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.664767  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.627959 14127 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:36.668203  9856 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 15 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:05:36.679157  9856 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 8
I20260709 13:05:36.679528  9856 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:05:36.670792  9271 rpcz_store.cc:276] 0709 13:05:34.787795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.041364 (+1253569us) service_pool.cc:224] Handling call
0709 13:05:36.042358 (+   994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.043506 (+  1148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.043548 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.048959 (+  5411us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.385546 (+336587us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:36.547369 (+161823us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.666482 (+119113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.666566 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.666628 (+    62us) raft_consensus.cc:1685] finished
0709 13:05:36.666651 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.666801 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7239,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":690,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":8470,"thread_start_us":616435,"threads_started":3,"wal-append.queue_time_us":461180}
W20260709 13:05:36.665946  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 129650us
I20260709 13:05:36.669512  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.680634  9855 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 15: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:36.680526  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:36.679694  9856 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 15 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:36.677481  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.679663  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:36.693616  9115 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2547) took 1299 ms (client timeout 1583 ms). Trace:
I20260709 13:05:36.660259 14005 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.699169  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.699561  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.699942  9122 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
W20260709 13:05:36.698932  9115 rpcz_store.cc:269] 0709 13:05:35.394095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.394203 (+   108us) service_pool.cc:224] Handling call
0709 13:05:36.693339 (+1299136us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54002f67d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:36.693515 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537070336}
I20260709 13:05:36.701198  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.660912 13854 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:36.687482 14081 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.179s	user 0.006s	sys 0.007s
I20260709 13:05:36.703200  9825 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:36.704674  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.713056  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:36.665483 14075 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:36.714109  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:36.714239  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 224385us
I20260709 13:05:36.716678  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.717674  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.718991  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.719847  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:36.721297 14132 log.cc:927] Time spent T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:05:36.722380  9122 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:36.677529 13996 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.734226  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.719173 13683 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:36.736137  9856 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:36.701222 14105 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.735675  9853 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
W20260709 13:05:36.742720 13684 consensus_meta.cc:287] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.514s	user 0.004s	sys 0.007s
I20260709 13:05:36.747228  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.748063  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.748534  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.748986  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.749267  9861 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 16: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:36.724089 14020 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:36.723882 14125 log.cc:927] Time spent T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.153s	user 0.001s	sys 0.001s
W20260709 13:05:36.751356 14133 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:05:36.713713 14005 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.748178  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.765583  9857 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:36.767455  9860 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:05:36.761497  9862 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:36.718314 13835 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:36.770140  9194 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
W20260709 13:05:36.771399 14124 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 13:05:36.773936 14128 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:05:36.769400 13835 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:36.775087 13835 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:36.777484  9855 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:36.752363 14024 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.793530 13996 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:36.794159 13996 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.794840  9122 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:36.795769  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.796342  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.799145 14005 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:36.804327 14065 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.872s	user 0.013s	sys 0.005s
I20260709 13:05:36.805676  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2802) took 2500 ms. Trace:
I20260709 13:05:36.806033  9266 rpcz_store.cc:276] 0709 13:05:34.304862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.306843 (+  1981us) service_pool.cc:224] Handling call
0709 13:05:34.307324 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:34.307810 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:34.307851 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:34.320337 (+ 12486us) log.cc:844] Serialized 26 byte log entry
0709 13:05:34.628130 (+307793us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:34.628173 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:34.628241 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.755358 (+2127117us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:36.805388 (+ 50030us) raft_consensus.cc:1685] finished
0709 13:05:36.805410 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.805617 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22069,"raft.run_cpu_time_us":1852,"raft.run_wall_time_us":25233,"spinlock_wait_cycles":110064896,"thread_start_us":267311,"threads_started":1,"wal-append.queue_time_us":462138}
I20260709 13:05:36.716722 13854 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.808750  9117 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:36.809401  9856 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:36.812969  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:36.722146 14075 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.814317  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.816205  9858 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.815171  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:36.816438  9117 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:36.814802  9123 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:36.819164  9117 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:36.820034  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.820232  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.796340 13996 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:36.848693  9114 raft_consensus.cc:2410] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:05:36.854185  9117 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:36.859114 13257 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.862380 13894 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.870501  9116 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2588) took 1335 ms. Trace:
I20260709 13:05:36.871002  9116 rpcz_store.cc:276] 0709 13:05:35.535469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.540310 (+  4841us) service_pool.cc:224] Handling call
0709 13:05:35.540588 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:35.577199 (+ 36611us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:35.641997 (+ 64798us) log.cc:844] Serialized 14 byte log entry
0709 13:05:35.967916 (+325919us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:35.967996 (+    80us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:35.968065 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.870228 (+902163us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b54001e3a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:36.870278 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.870434 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39380,"prepare.queue_time_us":156994,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":37026,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":7855,"spinlock_wait_cycles":83138944,"thread_start_us":330653,"threads_started":2,"wal-append.queue_time_us":363443}
I20260709 13:05:36.875717 13761 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.881968  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2576) took 1357 ms. Trace:
I20260709 13:05:36.882867  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2821) took 1350 ms. Trace:
W20260709 13:05:36.884203  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.886752  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2170) took 1780 ms. Trace:
I20260709 13:05:36.887149  9264 rpcz_store.cc:276] 0709 13:05:35.106398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.325176 (+1218778us) service_pool.cc:224] Handling call
0709 13:05:36.325561 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.325951 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.325988 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.333312 (+  7324us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.556584 (+223272us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:36.562875 (+  6291us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.879251 (+316376us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b18000597a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f7653571c0a 00007f765ccb1219 00007f76535f9a40 00007f76535f44e4 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287 00007f76566ff208
0709 13:05:36.879412 (+   161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.879493 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.879536 (+    43us) raft_consensus.cc:1685] finished
0709 13:05:36.879632 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.886691 (+  7059us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2660,"prepare.queue_time_us":7883,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":1459,"raft.queue_time_us":332141,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":4255,"spinlock_wait_cycles":5276544,"thread_start_us":184778,"threads_started":1,"wal-append.queue_time_us":311026}
I20260709 13:05:36.892928  9858 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:36.893174  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.894253  9860 raft_consensus.cc:2410] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:36.888486  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.895416  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.882475  9120 rpcz_store.cc:276] 0709 13:05:35.524731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.526443 (+  1712us) service_pool.cc:224] Handling call
0709 13:05:36.693498 (+1167055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.693695 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.693724 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:36.693771 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:36.693801 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.693865 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.881807 (+187942us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002f67d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:36.881911 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2683239168}
I20260709 13:05:36.883263  9267 rpcz_store.cc:276] 0709 13:05:35.532861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.418537 (+885676us) service_pool.cc:224] Handling call
0709 13:05:36.418798 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.430751 (+ 11953us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:36.470619 (+ 39868us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.882529 (+411910us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:36.882581 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:36.882630 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.882708 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.882814 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":282761,"prepare.queue_time_us":78579,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":4822,"raft.queue_time_us":11931,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":30488,"thread_start_us":130736,"threads_started":1,"wal-append.queue_time_us":234059}
I20260709 13:05:36.822633 13807 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:05:36.897015  9198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 905) took 2068 ms (client timeout 1968 ms). Trace:
I20260709 13:05:36.897289  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:36.898033  9859 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:36.899094  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.900813  9859 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:36.897446  9198 rpcz_store.cc:269] 0709 13:05:34.829031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:34.829211 (+   180us) service_pool.cc:224] Handling call
0709 13:05:36.896816 (+2067605us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b5400399254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:36.896944 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710295296}
I20260709 13:05:36.843106 13684 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:36.902382  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:36.902894  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:36.903215  9266 raft_consensus.cc:2410] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:05:36.905445  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 190089us
I20260709 13:05:36.906843  9235 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:36.907655  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:36.911296  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:36.912359  9263 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:36.836838 13980 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:05:36.916599  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2153) took 2578 ms. Trace:
I20260709 13:05:36.919624  9269 rpcz_store.cc:276] 0709 13:05:34.338624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.340585 (+1001961us) service_pool.cc:224] Handling call
0709 13:05:35.340875 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:35.343833 (+  2958us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:35.458666 (+114833us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.132556 (+673890us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b1800462520. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:05:36.675246 (+542690us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.703994 (+ 28748us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.704502 (+   508us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:36.715626 (+ 11124us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.907944 (+192318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.908003 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:36.916316 (+  8313us) raft_consensus.cc:1685] finished
0709 13:05:36.916349 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:36.916530 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":526531,"prepare.queue_time_us":115420,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":85,"raft.queue_time_us":23112,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":52855,"spinlock_wait_cycles":8640896,"thread_start_us":542329,"threads_started":2,"wal-append.queue_time_us":1118254}
I20260709 13:05:36.905979 13852 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:36.849819 14078 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.925047 13807 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:36.933450  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:36.934880 14151 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:36.935478 14151 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.937846 14151 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:36.927258 13684 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:36.942250  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.944175 13814 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.944393 13807 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.925521 13921 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:36.947585  9856 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:36.944962 14142 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:36.953804  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:36.956714  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:36.961992  9121 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:36.962889  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:36.964707 14145 log.cc:927] Time spent T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:05:36.964872 13683 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:36.971315  9198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:36.971935  9198 raft_consensus.cc:606] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:36.973960  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 293503us
W20260709 13:05:36.974947  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 50253.4us
I20260709 13:05:36.974695  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:36.928155 14115 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:05:36.940732 13478 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:37.003609  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:37.019429  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:36.998204 13659 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:36.929831 13852 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.031504 13814 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:37.031733 14156 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:37.032169 14156 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:37.032519 14156 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:37.034397 13824 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:37.000890 14151 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:36.995764 13977 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:05:37.045985 13898 raft_consensus.cc:993] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:05:37.046552 13898 raft_consensus.cc:1081] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:05:37.048715 14156 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.051116 14156 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.006668 13761 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:05:36.996253 13980 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:05:37.055837  9164 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:37.005255 14142 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:37.005864 13478 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:37.025452 14015 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:37.030391 13832 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:05:37.060686 14148 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:05:37.060746 13683 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:37.061200 13977 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.065693 14075 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.068841 14078 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.071691 13852 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.026010 14117 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:37.017586 14096 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:05:37.073395 13684 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.074749 13921 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.076363 14044 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:37.090322 13761 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:05:37.078529 13674 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:05:37.053767 14146 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:05:37.087723 14015 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:37.091576 13832 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:37.098505 14072 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:37.100660 14117 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:37.129974 14015 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:37.103819 13977 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.133066 14072 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:37.136257 14117 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:37.166601 14127 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.158869 14142 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:37.078135  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2302) took 1122 ms. Trace:
I20260709 13:05:37.063616  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2257) took 1459 ms. Trace:
W20260709 13:05:37.228965  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 209384us
W20260709 13:05:37.242390  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 235943us
I20260709 13:05:37.242892  9194 rpcz_store.cc:276] 0709 13:05:35.604006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.625331 (+ 21325us) service_pool.cc:224] Handling call
0709 13:05:37.063431 (+1438100us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400421dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:37.063554 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2559846144}
I20260709 13:05:37.244621  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:37.242439  9189 rpcz_store.cc:276] 0709 13:05:35.955677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.955849 (+   172us) service_pool.cc:224] Handling call
0709 13:05:36.628809 (+672960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.629034 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.629068 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:36.629116 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:36.629160 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.629236 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.077943 (+448707us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b54000a7150. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:37.078065 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546941696}
I20260709 13:05:37.257714  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2820) took 2435 ms. Trace:
W20260709 13:05:37.249606  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 274867us
I20260709 13:05:37.265100  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:37.251933  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:37.258265  9268 rpcz_store.cc:276] 0709 13:05:34.821726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.053340 (+1231614us) service_pool.cc:224] Handling call
0709 13:05:36.053684 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.053970 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:05:36.104995 (+ 51025us) log.cc:844] Serialized 14 byte log entry
0709 13:05:36.418594 (+313599us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b180029e6c0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:05:36.947073 (+528479us) log.cc:844] Serialized 14 byte log entry
0709 13:05:37.074248 (+127175us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b4800061ed8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:05:37.256624 (+182376us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:37.256778 (+   154us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:37.256874 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.257046 (+   172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.257402 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155621,"prepare.queue_time_us":56328,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":957,"raft.queue_time_us":208266,"raft.run_cpu_time_us":983,"raft.run_wall_time_us":100470,"spinlock_wait_cycles":69214976,"thread_start_us":681315,"threads_started":2,"wal-append.queue_time_us":774238}
W20260709 13:05:37.252723  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 191949us
I20260709 13:05:37.277801  9164 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:37.257012  9271 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:37.270128  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 169370us
I20260709 13:05:37.297899 13257 raft_consensus.cc:2764] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:37.300414  9268 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:05:37.308760 13232 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:37.311653 13257 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:37.125257 13854 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:37.347677  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49723.6us
I20260709 13:05:37.315791  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:37.110437 14044 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.128533 13478 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:37.380745 13478 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:37.382769 14127 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.133330 13832 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:37.389446 13601 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:37.393724  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 128922us
I20260709 13:05:37.396397 14157 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:37.397203 14157 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:37.397770 14157 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:37.108697 13674 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:37.400384 13674 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.402052 13674 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:37.402233 14072 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:37.407900  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.374810 13156 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:05:37.416345  9263 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:37.421363 13920 consensus_meta.cc:287] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.807s	user 0.009s	sys 0.003s
I20260709 13:05:37.422156 13920 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.423362 14044 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.411471 13922 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:37.429170  9267 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:37.429777  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.430516  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.427125  9164 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:37.424831 13156 consensus_queue.cc:1243] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:37.470171  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.471084  9268 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:37.473924 14072 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:37.476955  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 105080us
I20260709 13:05:37.478760 13659 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:37.480456  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.481973 13977 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.428319 13601 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:37.487393  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:37.491941  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 224453us
I20260709 13:05:37.502774  9268 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:05:37.512660  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:37.514199  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.515116  9264 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:37.505219  9263 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:37.516037  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:37.502530  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 504884us
I20260709 13:05:37.533236  9269 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:37.535876  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.537508  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:37.505637 13920 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.539137  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:37.540235  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.541628  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.516242 13601 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:05:37.545040 13981 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.550797  9272 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:05:37.561995 13934 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:37.563069  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:37.567245  9272 pending_rounds.cc:85] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 8
I20260709 13:05:37.567565  9272 pending_rounds.cc:107] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 9.9
W20260709 13:05:37.565883  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 60966.8us
I20260709 13:05:37.569212  9272 raft_consensus.cc:2889] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.570235 14005 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:05:37.571152 13817 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.567379  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:37.571439 13824 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:05:37.571703 13817 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.576428 13981 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.578904 13981 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:37.581254  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 60254.9us
I20260709 13:05:37.582253  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:37.591972  9122 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:37.573649 13232 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:05:37.599040  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:37.602196  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.604305  9270 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:37.603372 13232 consensus_queue.cc:1243] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:37.603375 13674 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:05:37.610630 14115 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:37.619985  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:37.618979 14115 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:37.623467 13934 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:37.626338 13934 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:37.619035  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2823) took 2038 ms. Trace:
I20260709 13:05:37.625612 14115 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:37.636258 13674 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:37.638468 14154 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.638s	user 0.013s	sys 0.020s
I20260709 13:05:37.641156 13817 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.641932  9164 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:37.642277  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 905) took 1193 ms. Trace:
I20260709 13:05:37.642782  9123 rpcz_store.cc:276] 0709 13:05:36.448745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.817370 (+368625us) service_pool.cc:224] Handling call
0709 13:05:36.817758 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:36.845827 (+ 28069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:36.845877 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:36.846427 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:05:36.918839 (+ 72412us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:36.918893 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:36.918980 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:37.636815 (+717835us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:37.641649 (+  4834us) raft_consensus.cc:1685] finished
0709 13:05:37.641676 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.642033 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23117,"raft.queue_time_us":25118,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":45770,"spinlock_wait_cycles":15179136,"thread_start_us":8977,"threads_started":1,"wal-append.queue_time_us":66927}
I20260709 13:05:37.632309  9265 rpcz_store.cc:276] 0709 13:05:35.533485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.427315 (+893830us) service_pool.cc:224] Handling call
0709 13:05:36.427728 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:36.429827 (+  2099us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:05:36.437579 (+  7752us) log.cc:844] Serialized 14 byte log entry
0709 13:05:37.074918 (+637339us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b4800061ed8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:05:37.500037 (+425119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:37.505072 (+  5035us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:37.522055 (+ 16983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.561237 (+ 39182us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.572278 (+ 11041us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194515,"prepare.queue_time_us":2127,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":52,"raft.queue_time_us":238252,"raft.run_cpu_time_us":971,"raft.run_wall_time_us":102423,"spinlock_wait_cycles":10086144,"thread_start_us":804089,"threads_started":2,"wal-append.queue_time_us":338779}
I20260709 13:05:37.648346 14103 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:37.649089 13601 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:37.651253  9825 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:37.651624 13601 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:37.652110 13601 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:37.653621 13601 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.616472 14015 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.655244 13601 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:37.655854  9266 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:37.656065 13854 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:37.657018 14015 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.646946 13674 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:37.649693 13981 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:37.654855  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:37.664566  9264 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:37.664438 13981 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.665340 14015 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:37.665526  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:37.666374 14015 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:37.667200 13981 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:37.670742 13674 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:37.675668 14005 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:37.676645 14005 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:37.678368 14167 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:05:37.681411 13156 consensus_queue.cc:1059] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:37.682415 14160 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:37.682518 13156 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:37.683243 13156 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:37.683528 14160 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:37.684069 14160 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:37.681938 14052 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:37.685180 13824 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:37.684209 13898 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:37.685684 13824 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:37.685959 13898 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:37.685364 14160 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.686975 13824 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.684278 13156 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.702566 13824 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:37.703001 13156 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:37.687110 13898 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.685601 13894 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:37.658713  9017 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 3 to 4, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 4 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:37.709888 13894 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:37.710165 13898 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:37.711218 13894 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.712293  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2603) took 1232 ms. Trace:
W20260709 13:05:37.699309  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 95481.1us
I20260709 13:05:37.740235  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1049) took 3215 ms. Trace:
W20260709 13:05:37.740468 13835 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.965s	user 0.011s	sys 0.009s
I20260709 13:05:37.740842  9272 rpcz_store.cc:276] 0709 13:05:34.525253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:35.635705 (+1110452us) service_pool.cc:224] Handling call
0709 13:05:35.652736 (+ 17031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.570735 (+1917999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.739794 (+169059us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b54000fa5d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:37.740107 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4363614464}
I20260709 13:05:37.687726 14160 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:37.760500 14052 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:37.752904 13659 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:37.741166 13835 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.791792 13835 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:37.772754 13981 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:37.792774 13981 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.712692  9115 rpcz_store.cc:276] 0709 13:05:36.480023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.836714 (+356691us) service_pool.cc:224] Handling call
0709 13:05:37.070831 (+234117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:37.071074 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:37.071381 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 13:05:37.487219 (+415838us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:37.497924 (+ 10705us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:37.503881 (+  5957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:37.506232 (+  2351us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:37.711975 (+205743us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400253d50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:37.712116 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9584,"prepare.queue_time_us":647,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":15897,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":163,"spinlock_wait_cycles":520772864,"thread_start_us":296943,"threads_started":1,"wal-append.queue_time_us":425938}
I20260709 13:05:37.794925 13981 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:37.784829 13832 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:37.787602  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 897) took 1441 ms. Trace:
I20260709 13:05:37.788729 14005 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:37.767998  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 100693us
I20260709 13:05:37.758062 13894 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:37.799616  9121 rpcz_store.cc:276] 0709 13:05:36.346408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:36.749619 (+403211us) service_pool.cc:224] Handling call
0709 13:05:37.763280 (+1013661us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540034d854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:37.787365 (+ 24085us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293706112}
I20260709 13:05:37.809820 13832 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:37.811000  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.813000  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:37.821934 13683 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:37.822767 13683 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.824213  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:37.821949 14117 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.827421 13832 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:37.827904  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 211657us
I20260709 13:05:37.828146 13832 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.828766  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:37.829792 14005 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:37.830341 13832 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:37.830943  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:37.834161  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:37.839869 13980 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:37.840492 13980 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:37.840883 13980 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:37.845657  9017 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:37.849956  9017 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:37.828846 14117 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.860095 13981 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:37.861069  9189 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:37.864105 13835 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:37.864955 13854 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:37.898073 13981 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.867447 13814 consensus_queue.cc:1059] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:37.867872  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:37.855348 13683 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:37.874162 14115 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.779309 14157 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:37.880931  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:37.885103  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:37.887758  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:37.895983  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:37.898101  9189 raft_consensus.cc:606] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:37.915261  9194 raft_consensus.cc:2410] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:05:37.921064  9189 raft_consensus.cc:614] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:37.924587  9263 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:37.925545  9267 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 6
I20260709 13:05:37.926203  9122 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:37.927294  9270 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:37.925866  9267 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:05:37.931090  9267 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.931439  9267 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:05:37.931595  9267 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:37.925814  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:37.973913  9265 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:37.978225  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:37.980973  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 149810us
I20260709 13:05:37.991472  9198 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:38.000254  9114 raft_consensus.cc:2393] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:38.002277  9120 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:38.008533 13934 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.009874  9269 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:38.011094 13934 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:38.014886  9828 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.968s (SENT)
I20260709 13:05:38.017237 13898 consensus_queue.cc:1059] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:38.018013 13156 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.018651 13156 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:38.018973  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.019788  9189 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:38.020759 13156 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.021375 13156 raft_consensus.cc:3055] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:38.021437  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.022353  9192 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:38.023080 13156 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:38.023535  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:38.024883 13683 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:38.026098 13683 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:38.026505 13683 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:38.026814  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.025942 14117 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:38.027962  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.028267  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:38.028793  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48046.6us
I20260709 13:05:38.029250  9191 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:38.029125  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:38.030959  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.027429 13683 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.032912  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.033114  9191 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:38.034574  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.035235  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.035876  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:38.033767  9191 raft_consensus.cc:606] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:38.038467 13683 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.038818  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.039855  9193 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:38.024055 13156 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.009008 13981 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.043952  9198 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:38.049451  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.050338 13156 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:37.891124 13980 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:38.058084  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.063792  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:38.064895  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.067404  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:38.028367 13478 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:05:38.075840  9087 leader_election.cc:336] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.505s (SENT)
I20260709 13:05:38.077539  9192 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:38.080802  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:38.082568  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.065770 13981 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:38.081795  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.082765  9194 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:38.086617  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.087949  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.071542 13832 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:38.076076 13478 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:38.098631  9193 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:38.100121  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.102568  9190 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:38.104660 13981 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.108596  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:38.110622 13981 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:38.122646  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 202806us
W20260709 13:05:38.145340  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.146611  9198 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:05:38.147149  9195 raft_consensus.cc:606] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:38.147629  9195 raft_consensus.cc:614] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:38.156136 13854 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:38.156774 13854 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:38.156834  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.157965  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.158650  9194 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:38.158216 13854 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.145779 13601 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.151134  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:38.102694 13817 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:38.196380  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.203219  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.203878 13852 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:38.204114  9198 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:38.203413  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:38.204593 13852 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:38.206141 13824 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.503s	user 0.006s	sys 0.012s
I20260709 13:05:38.168697 13854 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:38.204061  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:38.169716 13832 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.213543  9120 raft_consensus.cc:2410] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:05:38.200433 14157 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:38.172071 14103 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:38.215178  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.231451  9115 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:38.236176  9190 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:38.237313  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.240865  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:38.243176  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 167403us
I20260709 13:05:38.245280  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:38.263754  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.264503 13257 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:05:38.271979  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:38.272661  9114 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:38.215328 13817 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:38.260077 13934 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:38.222749 13478 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.283653 13804 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:05:38.270031 14103 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.271104 13805 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:05:38.286633  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.270376 13980 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:38.256299 14115 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:38.288798  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.267818 13824 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:38.320010 13824 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.320508 13824 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:38.321492 13805 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:05:38.290882  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.292304  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.294694 13981 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:38.323081 13981 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:38.323654  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.324481  9197 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:38.324292 13981 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.324887 14182 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:38.325855 13981 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:38.250988 13852 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.271183  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:38.300546  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:38.305352  9189 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
W20260709 13:05:38.306633  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.310851  9193 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:38.267433 13835 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:38.251329 13832 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.312359  9114 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:38.317040  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.317623  9196 raft_consensus.cc:2393] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:38.318444  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.296109  9190 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:38.325553 14182 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:38.326386  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.325419  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.329627  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.330021  9198 raft_consensus.cc:2410] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:38.331117  9192 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:38.320043  9119 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:38.338981 13478 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:38.346822  9197 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:05:38.347889  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.348400  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.349100  9189 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:38.354668 14182 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.337893 14015 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.356248 14015 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.358510 14015 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:38.359903 13835 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.362895 13835 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.368085 13835 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.368851 13835 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.371939 13817 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:38.371910 13835 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.374079  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:38.374845  9856 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 16: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:38.374972  9161 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:38.375985 13807 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.376542  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:38.376791  9189 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:38.360582 13832 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:38.383299  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.382529  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.393781  9192 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:05:38.398710  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:38.399407  9856 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:38.402004 14185 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:38.402815 14185 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.404098  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.404536 14183 raft_consensus.cc:3055] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:38.404863  9123 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:38.404994 14183 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:38.405752 14185 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:38.406244 14183 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.406777 14103 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:38.408702  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:38.409130 14183 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:38.386410  9162 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:38.410473 13934 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:38.410905 13934 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:38.411273 13934 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:38.412385 13156 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:38.417023 13824 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.395274 14115 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:38.427567 14160 raft_consensus.cc:993] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:05:38.441079  9235 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:38.419310 13934 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.451846  9192 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:38.452706  9192 raft_consensus.cc:606] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:38.453130  9192 raft_consensus.cc:614] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:38.456655 14183 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:38.465900 13894 raft_consensus.cc:1081] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:05:38.467959 13832 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:38.466521 14115 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:38.499343  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.485320  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:05:38.515552  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 262728us
I20260709 13:05:38.492103 13832 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:38.529333  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.526803  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:38.545404 14184 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:38.546240 14184 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.549353 14184 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.552345  9828 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:38.553131 14103 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:38.553913 14103 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:38.555044  9828 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:38.556949  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:38.557982  9855 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:38.558561  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:38.552544 13824 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:38.554980 14103 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.564646 14189 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.572866  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.572536 14115 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.573462  9192 raft_consensus.cc:2376] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16. Re-sending same reply.
I20260709 13:05:38.574275  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.574460  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.575130  9194 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:38.573480  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.531781 13832 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:38.577322  9198 raft_consensus.cc:2410] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:38.575151  9189 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:38.580713  9120 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:38.581002  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.581087 13934 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:38.581943  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.582311  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:38.582963  9123 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:38.583280  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.584096  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.556002  9268 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:05:38.588310  9268 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:38.588913  9268 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.511161 14183 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.593884  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:38.591411  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47348.9us
I20260709 13:05:38.599999 13601 raft_consensus.cc:3055] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:38.600487 13601 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:38.601459  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.601601 13601 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.602171  9194 raft_consensus.cc:2376] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12. Re-sending same reply.
I20260709 13:05:38.602787  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.595716  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:38.604338  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.605074  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:38.607132  9189 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:38.614481  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:38.623469  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:38.630752  9855 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:38.633391  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.632814  9198 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
W20260709 13:05:38.644266  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.616312 14189 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:38.621497 13981 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:38.650507  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.613384 13894 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:38.652328  9193 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:05:38.657315  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 102361us
I20260709 13:05:38.626883 14184 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:38.653635 13894 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:38.658766 14184 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.661415 14184 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:38.668838  9859 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:38.673719 14183 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.649067 13601 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:38.687054  9828 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:38.697935  9189 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:38.708313 14183 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:38.694213 13832 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:38.699301  9123 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:38.701078  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:38.702100  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 185233us
I20260709 13:05:38.702478  9089 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:38.703692  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.717311  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.720959  9087 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.189s (SENT)
I20260709 13:05:38.724157  9828 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:38.726001 13981 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:38.726449 13981 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:38.726878 13981 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:38.727921  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2189) took 1212 ms. Trace:
I20260709 13:05:38.728341  9267 rpcz_store.cc:276] 0709 13:05:37.515488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:37.646924 (+131436us) service_pool.cc:224] Handling call
0709 13:05:37.925259 (+278335us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:05:38.074761 (+149502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:38.074786 (+    25us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:05:38.093068 (+ 18282us) log.cc:844] Serialized 78 byte log entry
0709 13:05:38.507231 (+414163us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:38.507278 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:38.507358 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:38.665605 (+158247us) raft_consensus.cc:1685] finished
0709 13:05:38.665636 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:38.715519 (+ 49883us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54002a3d50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:38.727847 (+ 12328us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23988,"raft.queue_time_us":29202,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":4322,"spinlock_wait_cycles":638953856,"thread_start_us":338917,"threads_started":1,"wal-append.queue_time_us":513517}
I20260709 13:05:38.730508 13894 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.725193 14183 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:38.743695  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.745492  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.747036  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:38.748188  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:38.719758 13898 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:38.752434  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:38.749763  9859 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 6
I20260709 13:05:38.757797  9859 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 10.7
W20260709 13:05:38.759124  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 576107us
I20260709 13:05:38.760556 13156 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.761160 13156 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:38.764998  9854 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:05:38.765401  9854 pending_rounds.cc:85] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 11
I20260709 13:05:38.765466 13156 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:38.705194 14189 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:05:38.765663  9854 pending_rounds.cc:107] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:05:38.766971  9854 raft_consensus.cc:2889] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:38.769284  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 64989us
I20260709 13:05:38.748296 14183 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 16 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:38.779258  9122 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:38.758137  9859 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 15 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:38.782615  9859 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:05:38.783006  9859 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 15 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:38.785684 14202 log.cc:927] Time spent T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:05:38.749094 14200 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:38.796226  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 57443.3us
I20260709 13:05:38.797017  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:38.797780  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:38.800901  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52854.1us
I20260709 13:05:38.760761 14196 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:38.751165 13804 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 13:05:38.792515 13854 consensus_meta.cc:287] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.558s	user 0.005s	sys 0.008s
I20260709 13:05:38.807000  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:38.796775 14200 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.810825 13805 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Committing config change with OpId 8.6: config changed from index -1 to 6, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:05:38.814253  9268 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:38.815834 14200 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:38.816032  9087 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.874s (SENT) [suppressed 40 similar messages]
I20260709 13:05:38.819528 14189 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:38.819999 14189 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:38.820305 14189 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:38.821053  9191 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Committing config change with OpId 8.6: config changed from index -1 to 6, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:05:38.824216  9119 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:38.828519  9121 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:38.834908  9268 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:38.806542 14181 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:38.846361  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.845341  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.853904  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.828729 13832 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:38.859926  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:38.861151 14115 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:38.828421 14196 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:38.849187 14181 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:38.877276  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:38.882345  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.888234  9853 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:05:38.894505  9826 leader_election.cc:336] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.612s (SENT)
I20260709 13:05:38.895243  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:38.855911 13854 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:05:38.872004 14115 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:38.894768 14181 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:38.900801 14184 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:38.920332  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:38.864571 14183 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.914635 14200 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:38.934854  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:38.937716  9853 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
W20260709 13:05:38.940900  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 133953us
W20260709 13:05:38.949312  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46438us
I20260709 13:05:38.950639  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:38.951023  9862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:05:38.953076  9855 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:38.941809  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:38.957692  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.940137 14200 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:38.961133 14206 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:05:38.961436 14215 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:05:38.961879 14207 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:05:38.962668 14208 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:05:38.963635  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:38.969039  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:38.978145  9857 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:38.970269  9860 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:38.984495  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:38.950006 13854 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:38.930326 14184 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.954707 14185 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:38.975696  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:38.992537 14215 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:38.998010 14215 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:38.998591  9860 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:05:39.000295 14201 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.234s	user 0.000s	sys 0.004s
I20260709 13:05:39.003497  9089 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:39.008862  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:05:39.011974  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.001672 14200 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:38.953181 14196 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.039773 13804 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:39.041570  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:39.018729 14103 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:39.043102 13824 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:39.043206 14103 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:39.043777 14103 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:39.044835  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.043680 13824 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:39.049465 13824 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:39.020953 14115 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:39.013664 13601 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:39.025578 14189 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.075631  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.028990  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:39.077122  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.077852  9859 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:05:39.078534  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 49653.3us
I20260709 13:05:39.032562 13170 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:39.077872 13832 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:39.012817 13854 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:39.033214  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.036473 13981 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.042652  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:39.082338 14015 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:39.082718 14221 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:39.082893 14015 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:39.083397 14225 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.037905  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.083895 14015 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.084067 14225 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.080416 13854 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.086602 14225 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:39.088276 13854 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:39.088544  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 99970.4us
I20260709 13:05:39.090641  9189 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:39.091044  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.091885 13832 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:05:39.092208  9860 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:39.078747  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.093267  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:39.093987  9189 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:39.094669  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.096908  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.099040  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:39.092471  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.083346 14221 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.105568  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:39.107069  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 69265.5us
W20260709 13:05:39.101063  9826 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.507s (SENT)
W20260709 13:05:39.109534  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.107415  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:39.109553  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:39.110337  9195 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:39.111536  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:39.108443  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:39.116039 14221 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:39.119717  9118 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:39.120280  9114 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:39.120883  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.117854  9119 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:39.094780 14200 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.071604 13601 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:39.125977  9164 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:39.100841 13981 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:39.108409 13854 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:39.142491  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.118815 14225 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:39.123127 13934 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:39.123008 13852 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:39.140578 13601 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:39.150756 14214 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:39.162864  9854 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:39.141185 14052 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:39.150627 13980 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:39.154978 13835 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:05:39.169013  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 76670.9us
I20260709 13:05:39.171066  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:39.180562  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:39.180786  9189 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:39.181275  9198 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:39.156836 13683 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:05:39.174893  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.170904  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:39.157928 14189 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:39.185092  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:39.140961 14200 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.179912 13980 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:39.195946 13980 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:39.196033  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.196174  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:39.196753  9263 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:39.196859  9192 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:39.197402  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.198688  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.199636  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.167562 14214 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:39.200420 14214 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:39.151114 14181 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.202760 14181 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:39.190344 14211 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:05:39.204600 14196 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.205161 14196 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.206473 14211 consensus_queue.cc:1243] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer 94504d29c6744f38bb06943479df9466 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:05:39.207562  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.173565 13934 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:39.213932 13934 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:39.215548  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.217662  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.217935  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.218250  9264 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:39.219201  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.219817  9267 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:39.220896  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.221679  9270 raft_consensus.cc:2410] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:39.222517  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.223130  9272 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:39.224174  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.225914  9264 raft_consensus.cc:2410] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:39.226485  9855 raft_consensus.cc:3055] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:39.198251  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.226850  9855 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:39.227289  9266 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:39.193197  9854 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:39.194036  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.228155  9268 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
I20260709 13:05:39.195055  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.203962  9857 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:39.229485  9271 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.231503  9193 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:05:39.234724  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.235354  9272 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 7.
W20260709 13:05:39.235807 14204 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.467s	user 0.005s	sys 0.001s
I20260709 13:05:39.234746  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.232161 13832 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.240542 13832 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.242537 13832 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:39.245990 14217 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:05:39.252239  9855 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.253515  9855 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.257906  9265 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:39.262547  9192 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:39.263926  9118 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:39.265728  9120 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:39.266211  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.267125  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.268008  9265 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:05:39.264547  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 116488us
I20260709 13:05:39.273506  9271 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 17: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.275830  9855 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:39.279656  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.282846  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.285889  9195 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:39.287577  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:39.288190  9192 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
W20260709 13:05:39.280015 13814 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:05:39.300086 14228 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.300803 14228 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.303678 14228 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:39.305627 14184 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:39.306032 14103 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.303650  9855 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:39.307983  9122 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:39.263409 14229 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.142s
I20260709 13:05:39.312443  9195 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:39.315915  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.316676  9265 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:39.317601  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.318315  9265 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.318645  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.320567  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.320936  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.319202  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.321751  9265 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:39.322124  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.322197  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.325114  9267 raft_consensus.cc:2393] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.294138 14200 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:39.326769  9114 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:39.320657  9271 raft_consensus.cc:2410] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:05:39.322556  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.330089  9263 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.330523  9265 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.344098  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.345021  9267 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 17: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.346153  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:39.346534 14203 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.340s	user 0.002s	sys 0.001s
I20260709 13:05:39.346828  9267 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:39.352221  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.352941  9855 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:39.322942 13824 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.358299  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.358381  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.359833  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.363626  9119 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:39.338012  9161 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:39.368049  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.370473  9266 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:39.380103  9235 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:39.328579 14196 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:39.382891  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.382778  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:39.391245  9266 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:39.390062  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.358238 13824 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:39.392222  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.393492  9271 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:39.395170  9265 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.395164  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.396667  9267 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.382426 14052 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:39.402426  9266 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:39.391098 13832 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:39.404610  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.406745  9271 raft_consensus.cc:2393] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.404842  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.395146 13894 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:39.382658 13170 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:05:39.400672 14052 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.409830  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.413129 13894 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.413674 13683 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.414073 13832 raft_consensus.cc:697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:39.434044  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.434741  9264 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:39.420324 14052 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:39.428179  9089 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:39.428390 14235 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:05:39.426517 13894 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:39.429242  9087 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.511s (SENT)
I20260709 13:05:39.458581  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:39.464370  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 72016.1us
I20260709 13:05:39.468180 14244 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:39.433475 13683 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.473076 13805 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:39.478475  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49251.9us
W20260709 13:05:39.480016  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 51942.9us
I20260709 13:05:39.481026  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:39.481971  9087 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.530s (SENT)
I20260709 13:05:39.482497  9087 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:39.484220 14237 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:39.484480 14243 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:39.484879 14237 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:39.486076 14237 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:39.472680 14244 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:05:39.502887 13683 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:39.503328 13601 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.507853  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.508520  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:39.509773  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 150788us
I20260709 13:05:39.511049  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.511955 13804 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:39.513247 14103 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:39.514055 14103 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:39.511957  9854 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:39.512434  9115 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:39.505964 13601 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:39.484946 14243 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.514966 13980 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.536062  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.540922  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.548555  9116 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:39.550186  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.551201  9265 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:39.553911  9860 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
W20260709 13:05:39.556768  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.556646  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.560322  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:39.563862  9826 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.982s (SENT)
I20260709 13:05:39.571694  9116 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:39.573525  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.575850  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:39.576036  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49943.2us
I20260709 13:05:39.481882 13832 consensus_queue.cc:237] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:39.580950  9861 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
W20260709 13:05:39.581842  9087 leader_election.cc:336] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.634s (SENT)
I20260709 13:05:39.523984 14243 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:39.579282  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.624291  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:39.523926 14200 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:05:39.632122  9854 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:39.542448 14234 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:05:39.620577  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:39.583163 14195 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.124s
I20260709 13:05:39.707183 14230 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.707783 14230 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:39.706858  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 142991us
I20260709 13:05:39.708896  9862 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:05:39.709537  9826 leader_election.cc:336] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.716s (SENT)
W20260709 13:05:39.709966 14226 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.510s	user 0.001s	sys 0.001s
I20260709 13:05:39.710196  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:39.710928  9862 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:05:39.711432  9826 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.760s (SENT)
I20260709 13:05:39.712293 14181 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:39.721814  9854 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:39.718725  9862 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.580061 14215 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.125s
I20260709 13:05:39.582800 14211 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:05:39.725772 14201 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:05:39.771337  9862 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:39.779032  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.777431  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:39.797318  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 85906us
I20260709 13:05:39.798117  9269 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 17: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.798581  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.634215 14214 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.777740 14181 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.837821  9860 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:39.839942  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.840708  9862 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:39.843596  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 241483us
I20260709 13:05:39.850088  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.819175 14244 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:05:39.854585  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.855819  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.856551  9862 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:39.856828 14195 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.857115 13980 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:39.857296  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.857378 14195 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:39.858724  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.859390  9862 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:39.861896  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.862735 14230 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:05:39.863037  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.864243  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.865689  9123 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:39.867619 14214 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:39.868085  9858 raft_consensus.cc:2393] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:39.869011  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.869663  9858 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:39.751909 13683 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:05:39.821112 14115 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:05:39.884032  9862 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:39.836336 14181 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.875185 14195 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:39.895653  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.895890  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.896454  9266 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:39.897097  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.898428  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.899652  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.900264  9264 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:39.903918  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.905305  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.906190 14230 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:39.906585  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.907536 14230 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:39.907701  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.908851 14230 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:39.890853  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 175924us
W20260709 13:05:39.891201  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:39.908296  9266 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:39.909397  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:39.916050  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.917461  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.918678  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.919699  9265 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.886873 14214 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:39.921422  9269 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.921350  9264 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:39.922763  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.926234  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.929176  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.930166  9271 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:39.930732  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.933198  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.934129  9265 raft_consensus.cc:2376] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13. Re-sending same reply.
I20260709 13:05:39.925284  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:39.937629  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.923035 13934 raft_consensus.cc:2764] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:05:39.939512  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.936867  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.926518 14195 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:39.939407  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.938311  9266 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:39.941546  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.949580  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:39.945916 13980 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:39.952265 13980 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:39.952716 13980 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:39.953311  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:39.954584  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:39.955178  9861 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:39.945587  9828 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:39.964277  9267 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:39.926769 14214 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.967842  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:39.974565  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 392688us
W20260709 13:05:40.013535  9087 leader_election.cc:336] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.655s (SENT)
I20260709 13:05:39.988078  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:39.979954  9266 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
I20260709 13:05:40.015599  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:40.016533  9266 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:40.012286  9271 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:40.020623  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:40.023247  9861 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
W20260709 13:05:40.031313  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 186884us
W20260709 13:05:39.973878 13894 consensus_meta.cc:287] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.004s
I20260709 13:05:39.988426 14200 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:40.014506  9269 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:40.049115 14103 raft_consensus.cc:1064] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:05:40.057475 14103 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.059053  9268 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:40.049538 13894 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.065879 13894 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:40.067021  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:40.067660  9268 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:40.068041 14052 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.001s
I20260709 13:05:40.068475 14052 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.070474 14052 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:40.071796  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:40.072383  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:40.073066  9272 raft_consensus.cc:2376] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12. Re-sending same reply.
I20260709 13:05:40.077080 14214 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:40.078994  9268 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:40.050019 14200 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:40.023537 14246 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.258s	user 0.004s	sys 0.002s
I20260709 13:05:40.035123 14237 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:05:40.043992 14254 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.045013 14234 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:40.045607  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:40.046211  9087 leader_election.cc:336] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.667s (SENT)
I20260709 13:05:40.046999 14243 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:39.973966 13170 raft_consensus.cc:3055] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:39.982664 13601 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:40.084079  9089 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:40.072433  9269 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:40.082546 13478 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.081337 13805 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.097492 13156 consensus_queue.cc:1059] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:40.083858  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:40.097259 13478 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.051436  9266 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:05:40.085325  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 286494us
W20260709 13:05:40.084697  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.109413  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.111022  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:40.113806 13156 raft_consensus.cc:3055] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:40.114326 13156 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:40.083392 14181 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:40.118952  9114 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:40.120415 13156 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.092427 13170 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:40.084254 14255 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:40.097396 13894 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.130302  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:40.139328  9115 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:40.101778 14228 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:40.146579  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:40.149753 13156 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:40.144730 14267 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:40.153287 13478 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:40.152822  9192 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:40.118958 14252 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:40.130146 14233 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.141398 14255 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.156872  9856 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:40.156679  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:40.148614 13601 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:40.158375  9266 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:40.115382 13805 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.091177 14254 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:40.162595  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 116421us
I20260709 13:05:40.155642 14267 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.166625 14267 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:40.159106 14255 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.173803 14254 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:40.179112  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.180276  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.180459  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:40.181172  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 235689us
I20260709 13:05:40.181391 14211 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:40.181881  9828 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:40.183029  9828 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:40.183378 14185 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:40.183806 14185 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:40.184149 14185 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:40.184979 14215 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:40.185524 14215 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:40.186443 14215 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:40.189285  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 90979us
I20260709 13:05:40.189383 14183 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:40.190021 14183 consensus_queue.cc:1243] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:40.124212 14200 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.190800  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.180593  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.151153 13894 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.191826  9191 raft_consensus.cc:2410] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:05:40.195582  9269 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:40.181879 14211 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:40.200224 14211 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:40.157608 14228 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:40.203720  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 57257.8us
I20260709 13:05:40.190974 14200 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:40.194335  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:40.193284  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:40.205013  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.205351  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.208474  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:40.209321  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.211992  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.212203  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.211805 13835 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:40.214756 13980 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.214489  9194 raft_consensus.cc:2393] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:40.215040 14228 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.180392 14243 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:40.249749  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 45759.1us
I20260709 13:05:40.271696 14254 consensus_queue.cc:1059] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:40.272819 14275 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:40.273170  9087 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:40.224644 13814 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:40.209182 14267 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:40.274870  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 57665.8us
I20260709 13:05:40.250821 14181 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:40.251547 13805 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:40.251963 13835 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:40.276039  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.254758  9191 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:40.254834 14255 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.255797  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.211452 14183 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:40.156682 14252 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.211212 13170 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.161000 14233 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.263110  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.218168  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:40.216948  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:40.223989  9162 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:40.264918  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 55670.1us
I20260709 13:05:40.292098  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.221184  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:40.269306  9856 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:40.272876  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:40.216192  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:40.292876  9263 raft_consensus.cc:2376] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14. Re-sending same reply.
I20260709 13:05:40.273286 14254 raft_consensus.cc:3055] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:40.282286  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.253502 13894 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:40.287719  9087 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:40.289240  9115 pending_rounds.cc:85] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 11
I20260709 13:05:40.275359 14214 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:40.297115  9193 raft_consensus.cc:2393] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:40.297263  9115 pending_rounds.cc:107] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:05:40.297788  9115 raft_consensus.cc:2889] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:40.299328  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:40.311605  9192 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
W20260709 13:05:40.295894  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 101665us
I20260709 13:05:40.302817 14275 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:40.305117 14249 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.581s	user 0.001s	sys 0.002s
I20260709 13:05:40.305403 13601 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:40.306521 13170 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:40.307680 14261 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:05:40.310791  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 1077) took 1013 ms. Trace:
W20260709 13:05:40.297623  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 73755.9us
I20260709 13:05:40.286674 14229 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:40.315608  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.296682 14254 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:40.317322 13980 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:40.313185 13835 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:40.331722  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.332144  9862 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 15 LEARNER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:05:40.332549  9193 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:40.319414 14252 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:40.316277  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.313954  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:40.288412 14243 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:40.324802  9194 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:40.322638  9270 rpcz_store.cc:276] 0709 13:05:39.297106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.304173 (+  7067us) service_pool.cc:224] Handling call
0709 13:05:39.347797 (+ 43624us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:05:40.216832 (+869035us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfe9c 00007f76535bfa5b 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:05:40.310704 (+ 93872us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77332736}
I20260709 13:05:40.283762 14267 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.326102 14229 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:40.324173 13805 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:40.273850 14185 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.330576 14181 raft_consensus.cc:697] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:40.317746 14255 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:40.331163  9162 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:40.295406  9195 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:40.330935 13835 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:40.364334  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.336530  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:40.337553 14214 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:40.338546  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.350654  9193 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:40.350970  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.351229 14278 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:40.383783  9194 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:40.351325  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.351586  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.383397  9862 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.357437  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:40.357857  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:40.387784  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49323.7us
I20260709 13:05:40.388996  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:40.379532  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 43888.7us
I20260709 13:05:40.382480  9191 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:40.365553  9189 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:40.390908  9238 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.406499  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:40.407769  9192 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 8.
W20260709 13:05:40.407631  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 91425.6us
I20260709 13:05:40.409631  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.343396 14196 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:05:40.351998 13805 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:40.339416 14261 consensus_queue.cc:1059] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:05:40.416689  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.379107 14048 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:40.352742 14243 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.417387 13156 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:40.452524 13156 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:40.415470  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:40.384881 14214 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:40.342950 14200 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:05:40.441008  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:40.461236  9860 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:40.449368  9192 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:40.406175 13894 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:40.455077  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:40.400727 14260 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:40.411377 13852 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:40.466053  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:40.417062 13814 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:40.417948 14230 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:40.470705 14267 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:40.429638 14276 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:40.446583 13601 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:40.476358 13805 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.375607 13835 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.480506 13156 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:40.480415 14229 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.460180 13832 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:40.425385 14254 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.379717 14181 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.542008  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2884) took 1182 ms. Trace:
I20260709 13:05:40.594686  9267 rpcz_store.cc:276] 0709 13:05:39.359530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.420358 (+ 60828us) service_pool.cc:224] Handling call
0709 13:05:39.421578 (+  1220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:39.964815 (+543237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:40.541764 (+576949us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54001c61d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:40.541951 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247459712}
W20260709 13:05:40.559636  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 168432us
W20260709 13:05:40.507429  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.505748  9862 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:40.537007 13852 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:40.467482 13894 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.464287 14048 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:40.599293  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:40.600154  9269 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:40.604200  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:40.607108  9087 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:40.611198  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.575132  9194 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:40.576516  9191 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:40.583420 13478 raft_consensus.cc:3055] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:40.619458 13478 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:40.586706  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:40.587074  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:40.621919  9263 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:40.593526  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:40.573549  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.624058  9123 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:40.624574  9192 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:40.573985  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.623602  9264 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:40.603818 13894 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:40.612957 14048 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.574340  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:40.627321 13601 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:40.575730  9193 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:05:40.627521  9195 raft_consensus.cc:2410] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:05:40.586117 14274 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:05:40.612349 13852 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.628980  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:40.582536 13835 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.540799 14265 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:40.641392  9198 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:40.646693  9267 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:40.647713 14276 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:40.589342 13814 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:05:40.650370 14195 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:40.566843 14243 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:40.568009 13832 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.653205 13478 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:40.600054 14244 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.579454 14254 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:40.651225 14185 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:40.661432  9162 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:40.683706 13478 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:40.685153  9162 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:40.666280  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 256748us
I20260709 13:05:40.687247  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.652340 14265 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.686380  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:40.690770  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 93201.5us
I20260709 13:05:40.694638  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:40.571712 14214 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.659942 14234 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:40.664968 13898 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.741014  9826 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:40.741088  9123 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:40.663610 14200 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:40.700229  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:40.666606 13832 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:40.652776 13601 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.771468  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:40.683722 14276 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:40.777269  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:40.690600 14265 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.707243 13980 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:40.789124  9122 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:40.698660 13683 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:40.781421  9826 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:40.709079 14115 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:40.746544  9860 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:40.786940  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:40.787204  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:40.665158 14195 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:40.801748  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 346750us
I20260709 13:05:40.828450  9119 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:40.846673  9123 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:40.848277  9117 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:40.795920 13980 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:40.770068 14236 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:05:40.824148 14215 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:40.827091 14115 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:40.793736 13683 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:40.771059 14230 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:05:40.764818 13898 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.814934 14278 raft_consensus.cc:697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:40.859151  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:40.689388 14244 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.863436 14214 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:40.787792 14276 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.784521 14234 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:40.869270 14211 consensus_meta.cc:287] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.002s
I20260709 13:05:40.871201 14233 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:40.877457 14282 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.113s	user 0.003s	sys 0.000s
I20260709 13:05:40.834465 14260 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:05:40.966439 14234 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:40.881417 14275 consensus_meta.cc:287] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.566s	user 0.005s	sys 0.005s
I20260709 13:05:40.837381 14228 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:40.967823 14275 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:40.967954 14228 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:40.968416 14195 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:40.968914 14228 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.969487 14278 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.885591  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:40.883872 14185 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:40.857457  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 68870.7us
I20260709 13:05:40.977229 14276 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:40.851027  9116 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:05:40.909197  9087 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.170s (SENT)
I20260709 13:05:40.980067  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:40.980847  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:40.981142  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 71996us
I20260709 13:05:40.981611  9859 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:40.982751  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:40.986197  9853 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:40.988234 14243 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:40.999863 14048 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.000607  9164 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:41.000515 14048 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.002278 14261 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:41.002897 14261 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:41.003168 14048 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.003993 14261 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:40.944152  9119 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:41.012915  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:40.949836  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 168507us
I20260709 13:05:40.855549 13980 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:40.949469 14183 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:40.892445 14115 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:40.884194 14215 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:40.884263 14252 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:40.925310 13683 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:41.017525  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:41.022056  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.030045  9121 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:39.578907  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:41.035523 13898 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:40.957413 14160 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:41.050745  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:41.054370  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.051486 14276 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:41.059544 13601 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.056913 14291 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:05:41.061266 14291 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:41.061281 14244 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.064090  9118 raft_consensus.cc:1240] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 8. Current term is 9. Ops: []
I20260709 13:05:41.027989 14215 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:41.080926  9855 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.009s	sys 0.005s
W20260709 13:05:41.081321  9855 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.009s	sys 0.005s
W20260709 13:05:41.087939  9828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.010s	sys 0.005s
W20260709 13:05:41.088371  9828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.010s	sys 0.005s
W20260709 13:05:41.088691  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 689982us
I20260709 13:05:41.089953  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:41.093816  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:41.094739  9264 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:05:41.008569 14211 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.100698  9117 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
W20260709 13:05:41.123087  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 351685us
I20260709 13:05:41.139619  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:41.079993 14183 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:41.147157  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.148814  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.149708  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.052469 14160 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:41.050518 14252 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.151533  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:41.151620  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 322537us
I20260709 13:05:41.156507  9853 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
W20260709 13:05:41.152472  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 101790us
I20260709 13:05:41.157313 13170 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.157696  9087 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:41.157781 13170 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.159137  9826 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:41.161870  9853 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:05:41.125187 14276 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:41.165350  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:41.185725  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.124471  9828 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:41.202046  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.153699 14291 consensus_queue.cc:1059] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:05:41.208756 13824 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:41.209342 13824 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.209759 13824 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:41.223780 14229 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:41.225389 14229 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.146867 14183 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:41.165169 13894 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.539s	user 0.007s	sys 0.002s
I20260709 13:05:41.239718  9238 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:41.152062 14052 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:05:41.240679  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 116247us
I20260709 13:05:41.241508 13170 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.152122 14234 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:41.257643  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 92353.6us
I20260709 13:05:41.238235 14275 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:41.261375  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 102320us
I20260709 13:05:41.265697  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:41.243418 14183 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.270488 14234 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.272650 14234 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.248260 14285 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:05:41.269724  9853 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:41.248874 14052 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.263882 13601 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 13:05:41.281164  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.256592 13804 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:41.254848 14214 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:41.283854  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:41.280172  9115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.010s	sys 0.015s
I20260709 13:05:41.254662 14276 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.289302  9115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.011s	sys 0.016s
I20260709 13:05:41.250171 13894 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.299257  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1406) took 1122 ms. Trace:
I20260709 13:05:41.301059  9238 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:41.286240 14052 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:41.304085  9855 rpcz_store.cc:276] 0709 13:05:40.177094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.327214 (+150120us) service_pool.cc:224] Handling call
0709 13:05:40.327524 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.327931 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:40.328207 (+   276us) log.cc:844] Serialized 14 byte log entry
0709 13:05:40.861883 (+533676us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b1800427480. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:41.081760 (+219877us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:41.119574 (+ 37814us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.119829 (+   255us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:41.119880 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.119970 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:41.293291 (+173321us) raft_consensus.cc:1685] finished
0709 13:05:41.299021 (+  5730us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.299190 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69779,"prepare.queue_time_us":1336,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":209367,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":212,"spinlock_wait_cycles":2917504,"thread_start_us":753024,"threads_started":2,"wal-append.queue_time_us":123582}
I20260709 13:05:41.287588 14285 raft_consensus.cc:697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:41.291342 13804 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:41.300230 14181 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:41.354743  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 343316us
I20260709 13:05:41.375526 13478 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:41.376153 13478 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.432678  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.434569  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.435815  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.438292  9861 raft_consensus.cc:2410] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:05:41.443006  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:41.339426 13804 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:41.446995 13478 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.297434 14214 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.389650 14236 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:41.464051 13170 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.464597 13170 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.375077 14300 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:41.465714  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:41.446463 13832 consensus_meta.cc:287] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.663s	user 0.005s	sys 0.004s
I20260709 13:05:41.297698 14276 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.477731 13170 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.388976 13934 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:05:41.482474 14115 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:41.430933 14244 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:05:41.448757 14211 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:41.521874  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:41.522348  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:41.522718  9198 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:41.523031  9193 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:41.523595  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:41.524075  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:41.495543 13156 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:41.427779 14261 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:05:41.455336 14290 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.524771  9189 raft_consensus.cc:2393] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:41.534508  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:41.466604 14283 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.042s	sys 0.046s
I20260709 13:05:41.353821 13824 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.454334 14200 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:05:41.484190 14185 consensus_meta.cc:287] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
W20260709 13:05:41.483659 14292 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 13:05:41.533360  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.455168 14306 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:41.469817 14229 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:41.513383 14265 consensus_meta.cc:287] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.001s
I20260709 13:05:41.511705 14236 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.569087  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.569764  9117 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:41.546715 13170 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.573638 13170 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:41.581633 13170 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:41.543934 14283 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.043s	sys 0.047s
W20260709 13:05:41.580302  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 279313us
I20260709 13:05:41.483633 13683 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:41.585500 14324 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:41.586213 14324 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.586807 13683 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:41.586623 14324 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.568919 14185 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:41.562382 14261 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:41.565883 14252 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.589847  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:41.566126 13824 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.568640 14200 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.512643 13832 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.575290 14265 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.614274 14214 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:41.615401  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:41.541373 14183 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.518654 14300 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.618611  9860 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:41.618928  9861 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:41.617722  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:41.621599  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 155951us
I20260709 13:05:41.578744 14254 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:05:41.619717 14261 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:41.641175 14261 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.643472 14261 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:41.646690 13832 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.646728  9161 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:05:41.650604 14314 log.cc:927] Time spent T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:05:41.650504 14052 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.651669  9195 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:41.653065 14265 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:41.653497 14304 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
W20260709 13:05:41.653502 14317 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:05:41.658715 14200 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:41.659063 14305 log.cc:927] Time spent T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.098s	user 0.000s	sys 0.002s
I20260709 13:05:41.534857 14285 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.659885  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.660604  9857 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:41.662137  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:41.663054 14319 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:41.663534 14319 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:41.663941 14319 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:41.665686 14183 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:41.546480 14290 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.667097 14276 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:41.625062 13894 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.667892 13824 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:41.668288  9858 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:41.672341  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 83533.7us
I20260709 13:05:41.673148 14290 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:41.673785 14309 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:41.673208 14267 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:41.674923 14267 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:41.671124  9270 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:41.641805 14300 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.684723  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:41.673393  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:41.687014  9194 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:41.687014  9193 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:41.690477 14265 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:41.696864 14215 consensus_meta.cc:287] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.575s	user 0.006s	sys 0.003s
I20260709 13:05:41.697340 14215 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.699638 14236 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.671080 14115 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:41.706904  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:41.690675 13898 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:41.710722  9858 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
W20260709 13:05:41.714962 14321 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:05:41.714663 14183 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:41.716545 14324 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.704250 14306 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:41.721740  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.722119  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:41.695387 14265 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.722944  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:41.723260 14307 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:05:41.724439  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.724819  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:41.725294  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.723994  9115 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2656) took 1117 ms. Trace:
I20260709 13:05:41.738760  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:41.740002  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:41.741539  9198 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:41.749185  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:41.749769  9123 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:41.742861  9191 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:41.749373  9193 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:41.749169  9115 rpcz_store.cc:276] 0709 13:05:40.592912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.593495 (+   583us) service_pool.cc:224] Handling call
0709 13:05:40.613480 (+ 19985us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:40.628094 (+ 14614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.628129 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:40.631840 (+  3711us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.289817 (+657977us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:41.289854 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.289925 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:41.658287 (+368362us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:41.701414 (+ 43127us) raft_consensus.cc:1685] finished
0709 13:05:41.701436 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.710325 (+  8889us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10406,"raft.queue_time_us":17312,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":16646016,"thread_start_us":622325,"threads_started":1,"wal-append.queue_time_us":828420}
I20260709 13:05:41.702929 14200 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 13:05:41.724462 14313 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:05:41.753314  9855 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:41.753585  9121 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:41.755179  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 48352.3us
I20260709 13:05:41.757004  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.757335  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:41.759438  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:41.760109  9856 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:41.760397  9193 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:41.760366  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:41.761896  9191 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:41.736202 14196 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.725306 13898 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:41.770066  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:41.771719  9198 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:41.762717  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:41.748083 13980 consensus_meta.cc:287] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.720s	user 0.003s	sys 0.008s
I20260709 13:05:41.770004  9191 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:41.774852  9193 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:41.777509  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:41.781975  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.747040 14236 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.752694 13814 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:41.783449 14196 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.788880  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:41.800614  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.798131  9856 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:41.864284  9855 raft_consensus.cc:2410] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:41.865473  9856 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:41.801298  9117 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:41.861492  9115 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:41.746873 13832 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:41.800463  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:41.863104  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:41.868382  9193 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:41.761214 14276 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:41.789994 14306 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:41.776539 13898 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:41.896515  9193 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:05:41.871366  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 145342us
I20260709 13:05:41.800143 14196 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:41.896468  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.723690 14267 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.898751  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:41.900193  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:41.900620  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 100196us
I20260709 13:05:41.904867  9193 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:41.800606 14324 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:41.903282  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:41.784010 14302 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:41.905361  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:41.890285 13894 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.862723 13814 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:41.871634 14200 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.920899  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:41.897693 13832 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:41.922751  9856 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:41.895821 14276 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:41.802136 13980 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.910413 14291 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:41.928694  9089 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:41.936412  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:41.937533  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1413) took 1142 ms. Trace:
I20260709 13:05:41.915071 14052 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:41.939599  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:41.871476 14236 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.938161  9862 rpcz_store.cc:276] 0709 13:05:40.784784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.784961 (+   177us) service_pool.cc:224] Handling call
0709 13:05:40.785279 (+   318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:40.830035 (+ 44756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:40.830086 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:40.830924 (+   838us) log.cc:844] Serialized 52 byte log entry
0709 13:05:41.302368 (+471444us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:41.302426 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.302517 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:41.722580 (+420063us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:41.920338 (+197758us) raft_consensus.cc:1685] finished
0709 13:05:41.920385 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:41.927326 (+  6941us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6052,"raft.queue_time_us":77417,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":80,"spinlock_wait_cycles":5094272,"thread_start_us":418132,"threads_started":1,"wal-append.queue_time_us":669645}
I20260709 13:05:41.928676 14276 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:41.950779 13824 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:41.951093 14230 raft_consensus.cc:3055] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:41.926913 14200 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.921595 14215 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:41.958027  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.988849  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.931516 13832 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.939322 14267 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:41.997200  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.997244  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.033260  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:41.906283 14255 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:05:41.925133 13894 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.958385 14291 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:41.927189 13814 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:41.988914  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.028191  9191 raft_consensus.cc:2410] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:05:41.998389 13934 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:05:42.034920  9116 raft_consensus.cc:2410] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:05:42.038116  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:41.958027 13804 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.027607 13824 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:42.045634  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 140338us
I20260709 13:05:42.048482 13814 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:42.025712 14230 raft_consensus.cc:740] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:41.940011 14324 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:05:42.083235  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:42.093328  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:42.084071 14328 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:05:42.084656 14333 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:05:42.085201 13683 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.088091  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.088589 14325 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.088455 14319 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.089422  9122 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:42.090485 13804 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.090730  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.082086  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.093600  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:42.115061  9196 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:42.095221  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.115866  9121 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:42.116438 13170 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:42.099063  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:42.101831  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.052228 13824 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:42.109899  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 192014us
I20260709 13:05:42.109939  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:42.118732  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.119979 14309 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:42.120669 14309 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:42.121903 14309 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.128849  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.131594  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.138110  9121 raft_consensus.cc:2410] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:05:42.080905 14306 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:42.139926  9119 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:42.135986  9089 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:42.144218  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:42.103561 13934 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:42.159034  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:42.159615  9195 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:42.103683 14215 consensus_queue.cc:1059] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:05:42.163156  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2896) took 1572 ms. Trace:
I20260709 13:05:42.165767  9270 rpcz_store.cc:276] 0709 13:05:40.585651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.586416 (+   765us) service_pool.cc:224] Handling call
0709 13:05:42.158006 (+1571590us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54004434d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:42.158129 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2414738176}
W20260709 13:05:42.158212 14335 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 13:05:42.161298 14345 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:05:42.180583 14345 consensus_queue.cc:1243] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:05:42.180806 14330 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:05:42.135815 13601 consensus_queue.cc:1059] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 13:05:42.182238 14338 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:05:42.182639 14319 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.127784 13683 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.185595  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.186774  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.179584  9195 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:05:42.188257  9161 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.188365 14230 consensus_queue.cc:260] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.192272 14052 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.196214 14052 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.197232 14306 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.198333 14052 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.192121 14260 raft_consensus.cc:3055] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:42.199576 14260 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:42.200382 14260 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.201123 13898 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.201560 14260 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:42.141072 14325 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.220286 14200 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.221585 14325 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.206012 13478 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:42.192111 13980 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.226233  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.205471 13852 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:42.227306  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.227897 14195 raft_consensus.cc:3055] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:42.228609 14195 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:42.229749 14195 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.231014  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.231591  9268 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:42.236150  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.236841  9270 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 5.
I20260709 13:05:42.236424  9018 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 9 to 11, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 11 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:42.237926  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.238430 14185 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.238250  9018 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: config changed from index -1 to 6, term changed from 8 to 11, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to <none>, NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added. New cstate: current_term: 11 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:05:42.239038 14185 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.209889 14236 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.241745 14348 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:42.242720 14348 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:42.242911 14185 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.243089 14348 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:42.243537  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.244351  9272 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:42.243673  9018 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:42.245342  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.245908  9264 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:42.245795  9018 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 3 to 9, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 9 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:05:42.249460 14320 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.685s	user 0.002s	sys 0.009s
I20260709 13:05:42.253201 14276 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.256130  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2587) took 1516 ms. Trace:
I20260709 13:05:42.257248 14265 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.256688  9114 rpcz_store.cc:276] 0709 13:05:40.739456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:40.741125 (+  1669us) service_pool.cc:224] Handling call
0709 13:05:40.741422 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.126930 (+385508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.126976 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:41.139563 (+ 12587us) log.cc:844] Serialized 26 byte log entry
0709 13:05:41.453422 (+313859us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:41.453462 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:41.453525 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:41.952318 (+498793us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:42.249145 (+296827us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b2400168cf8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765347d9c4 00007f765347bacb 00007f7653636dc9 00007f7653636a55 00007f765363609d 00007f765626810e 00007f765627f88c 00007f765627ebb6 00007f765627e7d8 00007f7656288221
0709 13:05:42.255849 (+  6704us) raft_consensus.cc:1685] finished
0709 13:05:42.255880 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.256057 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55317,"raft.run_cpu_time_us":3303,"raft.run_wall_time_us":24550,"spinlock_wait_cycles":21251072,"thread_start_us":639517,"threads_started":2,"wal-append.queue_time_us":383097}
I20260709 13:05:42.259497 13898 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.259670 13824 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.259534  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:42.261518  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.261732 13824 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.262316 13898 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:42.262645  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:42.263382 13898 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:42.263890 13898 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:42.264770 13898 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.266902  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:42.270015 14332 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.360s	user 0.002s	sys 0.000s
I20260709 13:05:42.271093  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.161659 14196 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:05:42.188957 14308 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:42.211488 13804 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:42.274151 13804 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.178143 14291 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.216800 14269 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:42.218117 13814 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:42.228012  9269 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:42.230367  9263 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:42.238545  9266 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:42.210080 14160 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.271844  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:42.176466 13934 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:42.303856  9162 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.278548  9164 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.209216 13832 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.210742 13894 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.280372  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.302569  9857 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:42.284803  9268 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:42.287169  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.288682  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.300207 13170 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:42.300756 14312 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:05:42.206967  9196 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:05:42.314023  9194 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:42.324625  9198 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:42.327411  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.325850  9267 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:05:42.351819  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 51271.1us
I20260709 13:05:42.351294  9264 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:42.325742 13170 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.352674  9265 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:42.359736  9266 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
I20260709 13:05:42.363546  9272 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:42.323463  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.364356  9270 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
W20260709 13:05:42.368949  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 105222us
I20260709 13:05:42.376950  9269 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:42.298039 13814 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:42.296505 14254 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.297497 14185 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:42.355836 14276 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:42.398761 13170 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.318091 14269 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:42.317830 14308 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:42.391698  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:42.392146  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 233192us
I20260709 13:05:42.368916 14325 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:42.422081  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:42.384253 14115 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:42.425168  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.429067 14229 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:42.429991  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:42.429841 14200 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:42.430852 14229 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:42.431649  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:42.432884  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:42.434953  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:42.454180  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.460109  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:42.465334  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 929) took 1023 ms. Trace:
W20260709 13:05:42.475049  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 356396us
I20260709 13:05:42.465885  9122 rpcz_store.cc:276] 0709 13:05:41.441430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.442532 (+  1102us) service_pool.cc:224] Handling call
0709 13:05:41.442818 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:42.114532 (+671714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.465010 (+350478us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b5400285d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:42.465253 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486530304}
I20260709 13:05:42.426385 14183 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.480666  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.486171  9122 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:42.425840 14325 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:42.486764  9122 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:42.433241 13804 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.436313 14229 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:42.491000  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.400429 14254 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.491379  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:42.426693 14308 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.530521  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.535074  9118 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:42.490804  9122 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:42.531605  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 153069us
I20260709 13:05:42.536167  9122 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:42.368224 14348 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.492324 14325 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:42.492246 14324 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:42.528506 14200 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:42.550743  9195 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:42.544121  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:42.544827  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 53913.3us
W20260709 13:05:42.561309  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 182169us
I20260709 13:05:42.565349  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.490487 13934 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:05:42.568863  9265 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:42.577831  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:42.578685  9195 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:42.585134  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.571926  9162 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.586450  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.559612 14261 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:42.558735 14260 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:42.603264  9268 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:42.603390  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:42.598057  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:42.599854  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:42.588594  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.605336  9194 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:42.600500  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:42.606027  9855 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:42.606417  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:42.606994  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:42.607609  9853 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:42.607018  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:42.611126  9856 raft_consensus.cc:2410] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 13:05:42.537690 13804 consensus_queue.cc:1175] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "94504d29c6744f38bb06943479df9466" responder_term: 14 status { last_received { term: 14 index: 7 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:42.617388 14254 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.572993 13832 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:05:42.572782 14265 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.595639  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.627564  9272 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:42.572597 14236 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:05:42.601403  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 72368.5us
I20260709 13:05:42.491031 14183 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.602023  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.640230  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.641060  9268 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:42.589113  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.595865  9267 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:42.563382 14302 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:42.621467  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:42.649772  9854 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:42.600863  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.637198  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:42.635161  9265 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:42.602849  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:42.654006  9264 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:05:42.627728 14350 log.cc:927] Time spent T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:05:42.538075 14269 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.657783  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:42.603878 14360 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:42.494742 13814 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.658869 14115 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:42.667550 14312 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:42.678402 14195 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:42.659077 14319 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:42.672714 14357 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:05:42.639834 14196 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:42.676885 14115 raft_consensus.cc:697] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:42.677929 14354 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 13:05:42.671294  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.684614 14308 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.666610 14367 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 13:05:42.689242 14337 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.636s	user 0.004s	sys 0.001s
I20260709 13:05:42.692799 14319 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:42.680049 14285 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:42.674597  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 85493.3us
I20260709 13:05:42.696529  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.681161  9272 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:42.768798  9162 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:42.760387 14351 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:05:42.754473  9857 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:42.755208  9855 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:42.759902 13805 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:05:42.763232  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 153960us
I20260709 13:05:42.635624 14339 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:05:42.761894 13824 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:42.764959  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 155850us
W20260709 13:05:42.766389  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.640934 14265 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.769937  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:42.773413  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.782604  9272 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:42.782837  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:42.792732 14183 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:42.792958 14367 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:42.770378 14319 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:42.793344 14367 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.793710 14367 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:42.774909  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.714389  9194 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:42.795917 13824 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:42.665573 14302 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.796108  9263 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:05:42.797103  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.796967 13824 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.798141  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.798434 14302 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.762676 13601 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:42.799209 13601 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.799705  9162 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.776247 13805 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:42.801350 13805 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.802601 13805 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:42.802266  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.803905 14312 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:42.804046  9270 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:42.804433 14312 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:42.799857 13852 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:42.805152 13852 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:42.805330 14312 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:42.806070 13852 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.788103 14115 consensus_queue.cc:237] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.773804 14285 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.809413 14285 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.773466 14308 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:42.798861  9263 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:05:42.810930  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.816933  9270 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:42.811242  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.816344 14302 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:42.822119 14302 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.824366  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1428) took 1186 ms. Trace:
I20260709 13:05:42.824613 14302 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.824908 14265 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.824803  9860 rpcz_store.cc:276] 0709 13:05:41.638010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.638220 (+   210us) service_pool.cc:224] Handling call
0709 13:05:41.638653 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:41.639173 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.639217 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:42.267333 (+628116us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b2c00047988. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:05:42.267699 (+   366us) log.cc:844] Serialized 26 byte log entry
0709 13:05:42.494383 (+226684us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:42.494426 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.494499 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:42.824086 (+329587us) raft_consensus.cc:1685] finished
0709 13:05:42.824106 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:42.824312 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":84380416,"thread_start_us":226300,"threads_started":1,"wal-append.queue_time_us":541415}
I20260709 13:05:42.826056 14368 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:42.826566 14368 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:42.819214  9263 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:42.824386 13601 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.826924 14285 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:42.828990  9862 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 9. Current term is 10. Ops: []
W20260709 13:05:42.831632  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 180676us
I20260709 13:05:42.835142  9272 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
W20260709 13:05:42.837533  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.840590  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.841763  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.827338 14195 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:42.826925 14368 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:42.875592  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.879045  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:42.881129  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.887916  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.900230  9194 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:42.888051  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:42.900753  9263 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:05:42.901365  9855 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:05:42.888808  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:42.902546  9860 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:42.889647  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.890439  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.892357 14368 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.894519 13832 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:42.895136  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.895658 14302 consensus_queue.cc:1059] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:42.896348  9858 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:42.897188  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:42.898554  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.917013  9857 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:42.921152  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.922960  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.923789 13601 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:42.924325 13601 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.834852 14285 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.926092  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.926805  9191 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:05:42.928004  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.929288  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:42.930645 14325 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.926884 14371 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:42.901882  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.904112  9195 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:42.933553 14371 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.906761  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.836390 13683 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.912596  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.915346  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:42.935907  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.936549  9117 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:05:42.936888  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:42.938174  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.944919  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:42.946821  9263 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:42.940900 13601 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.954298  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.926518 14302 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:42.940955  9857 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
W20260709 13:05:42.943442 14185 consensus_meta.cc:287] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.000s
I20260709 13:05:42.945812  9272 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:42.946009  9116 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:42.949358 14285 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:42.950313 14276 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:42.950593  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.950910  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:42.939643 14371 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.950253  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:42.952852  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:42.953102  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.940588 14360 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:42.955686  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:42.957979  9855 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
W20260709 13:05:42.975605  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:42.956166  9263 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:42.961329 13601 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:42.961934  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:42.955487  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 179973us
I20260709 13:05:42.963562  9825 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:42.971436  9860 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:42.897785 14195 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:42.980340 13601 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:42.985937  9194 raft_consensus.cc:1240] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 5. Current term is 6. Ops: []
I20260709 13:05:42.989959  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.990293  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.990614  9194 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:42.955919 14353 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:42.956050 13832 raft_consensus.cc:2764] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:05:43.003237  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 47642us
I20260709 13:05:42.964758 14185 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:43.005092  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:42.975603 14285 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:43.010456  9089 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:43.012411  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:42.974903 14376 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:43.013626 13601 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:42.956693 14367 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.022333 14195 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:43.023452 14367 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:43.013300 14360 consensus_queue.cc:1059] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:05:43.025213 14360 raft_consensus.cc:3055] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:43.025626 14360 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:43.026490 14360 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.027940 14360 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:43.029151  9122 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:05:43.029556  9122 pending_rounds.cc:85] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 6
I20260709 13:05:43.029772  9122 pending_rounds.cc:107] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:05:43.030002  9122 raft_consensus.cc:2889] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:43.019824  9263 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:43.034112  9263 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:43.036439 14325 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:43.041736  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:43.042150  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.007031 14309 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:43.043292  9263 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:42.957217 14302 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.050910 14302 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:43.012990  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:43.051908 13601 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:43.052438 13601 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:43.012789 14291 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:42.975167 14276 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.053825  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:43.019268 14371 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:43.053879  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:43.017915 14376 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.059054  9263 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:05:43.056696  9122 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:05:43.057313  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 238848us
I20260709 13:05:43.062577  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:43.063504  9116 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:43.053536 13601 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.071780  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:43.044819 14185 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:43.074354  9117 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:43.076633  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 274568us
I20260709 13:05:43.082599 14185 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.084260  9195 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:43.084815 14185 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:43.092051  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 104049us
I20260709 13:05:43.060264 14376 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:43.092092  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 128614us
I20260709 13:05:43.071834 14306 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:43.059868 14291 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:43.100348  9269 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:43.101315  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:43.081696 14371 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:43.077435 14308 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:43.112201  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.071902 14261 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:05:43.114903  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.119306  9269 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:43.114809  9828 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:43.116358  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:43.120460  9861 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:43.117012  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:43.121299  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:43.121614  9857 raft_consensus.cc:2393] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:43.124758  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:43.124874 14291 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.125541  9860 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:43.126478 14302 raft_consensus.cc:2764] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:05:43.112190 14115 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.129988 14115 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.131330 14319 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:43.133486 14319 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:43.135914  9860 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:05:43.136277  9860 pending_rounds.cc:85] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 10
I20260709 13:05:43.136502  9860 pending_rounds.cc:107] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:05:43.136737  9860 raft_consensus.cc:2889] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:43.138899  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.124724 14371 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.142266 14368 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:43.136685  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:43.146468 14349 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.714s	user 0.002s	sys 0.000s
I20260709 13:05:43.150597 14183 raft_consensus.cc:3055] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:43.151088 14183 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:43.153566  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 982) took 1238 ms. Trace:
I20260709 13:05:43.157488  9272 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:05:43.160180 14296 raft_consensus.cc:3055] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:43.160734 14296 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:43.161731 14296 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:43.163023 14296 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:43.167322  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.167652 14398 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:43.168231 14398 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.153998  9193 rpcz_store.cc:276] 0709 13:05:41.915626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:41.916004 (+   378us) service_pool.cc:224] Handling call
0709 13:05:41.916325 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:41.916682 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:41.916714 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:41.917234 (+   520us) log.cc:844] Serialized 52 byte log entry
0709 13:05:42.291364 (+374130us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:42.350057 (+ 58693us) log.cc:844] Serialized 14 byte log entry
0709 13:05:42.350286 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:42.350364 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:43.145857 (+795493us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:43.153212 (+  7355us) raft_consensus.cc:1685] finished
0709 13:05:43.153319 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:43.153473 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":58202,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":4232,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":4613,"spinlock_wait_cycles":84202624,"thread_start_us":264607,"threads_started":1,"wal-append.queue_time_us":441008}
W20260709 13:05:43.179070  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.184428 14183 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.186131 14183 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:43.187618 13814 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:05:43.188103  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.188894 14302 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:43.189389 14302 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.191558 14302 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:43.224175 14319 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:43.256592  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.124572  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:43.271150  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:43.275036  9120 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:05:43.302953  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:43.316787  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 201893us
I20260709 13:05:43.318672  9269 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
W20260709 13:05:43.323602  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.334280 14115 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:43.302812 14255 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:43.340065  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:43.340709  9193 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:43.340761 13478 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.341399 13478 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.342206 13170 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:43.343168  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.343727  9270 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:43.344592  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.345965  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:43.350565  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:43.353106  9270 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:43.350204  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.358534  9193 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:43.355700  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:43.367961 13478 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:43.337563 14230 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:43.370639  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:43.340085 14255 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.340245 14115 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:43.347805 14369 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:43.352537 14326 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:43.269225 14308 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.384013  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.322939 14399 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:05:43.389127  9193 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:05:43.400920  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:43.403242 14393 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 13:05:43.404824 14397 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:05:43.405956 14390 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
W20260709 13:05:43.411921  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 94922.9us
W20260709 13:05:43.413458  9826 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 2.107s (SENT)
I20260709 13:05:43.413841  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.414068  9862 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:43.378911 14255 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:43.423906  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:43.432633  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:43.440963 14348 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:43.442513  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:43.400729 14326 raft_consensus.cc:697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:43.443838  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:43.443632 14326 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.448930  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:43.449352  9267 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:43.451223  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:43.455778  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 100141us
I20260709 13:05:43.457753  9269 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:05:43.462929  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:43.463625  9860 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:43.463419  9862 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:43.389856 13683 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:43.469477  9164 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:43.473071  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:43.475045  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.475883  9269 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:43.480640  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.481137  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.481436  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.481704  9272 raft_consensus.cc:2376] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13. Re-sending same reply.
I20260709 13:05:43.481817  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.482157  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.482822  9271 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:43.483006  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.483533  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:43.483810 14412 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:43.484148  9119 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:43.484318 14412 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:43.390106 14276 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:43.485175 14412 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.390192 14115 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.486568 14276 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:43.478291 14358 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:05:43.488396 14115 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:43.401075 14398 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:43.468372 14265 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:05:43.491341 14369 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:43.491755 14369 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.492090 14369 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:43.492825 13898 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:43.495019 13898 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:05:43.496500  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.497483 13898 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:05:43.511363  9265 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:43.499991  9862 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:05:43.501762  9190 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:05:43.512995  9862 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 9
I20260709 13:05:43.513324  9862 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 15.10
I20260709 13:05:43.513702  9862 raft_consensus.cc:2917] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Aborting config change with OpId 15.10: Aborted: Op aborted by new leader
I20260709 13:05:43.514245  9862 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 16 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:43.506244  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.509074  9263 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:05:43.504442  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:43.509735  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:43.516878  9858 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:43.497627  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.503520  9193 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:43.500154 14360 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:43.535851  9195 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:05:43.539776  9198 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:05:43.542940  9195 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:43.543820 13170 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:43.548677 13478 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:43.551879  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:43.563141  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:43.510624 14267 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:43.565128  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:43.568399  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:43.571084  9120 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:43.571862  9859 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:05:43.453373 14269 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 13:05:43.572329  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:43.576576  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:43.581259  9861 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:05:43.572425  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:43.583216  9855 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:43.573946  9860 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:05:43.518010 13683 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:43.523003 14265 consensus_queue.cc:1243] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:43.569521  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:43.591394 13601 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:43.591924 13601 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:43.542511 14261 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:43.589804  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:43.595849  9854 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:43.596634  9859 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:05:43.604672  9087 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:43.606662  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:43.608259  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:43.630336  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:43.616050 13601 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.653149  9116 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:05:43.655049 14376 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.005s
W20260709 13:05:43.655130 14185 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.563s	user 0.006s	sys 0.010s
I20260709 13:05:43.656715  9858 raft_consensus.cc:2410] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:05:43.636427 14276 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:43.605515 14261 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.575898 14371 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:05:43.616039 14052 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:43.610411 13980 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:05:43.612437 14311 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:43.617484 14306 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:43.688053  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.611965 13824 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:43.644393 14355 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:05:43.751618  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:43.759810  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 58056.9us
I20260709 13:05:43.689579 14376 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.750365 14311 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:43.757169 13814 consensus_meta.cc:287] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.569s	user 0.006s	sys 0.009s
W20260709 13:05:43.796811  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 231738us
I20260709 13:05:43.696820 14261 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:43.697685 14052 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:43.783716 14416 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:43.692402 14291 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:05:43.695297 14185 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.789177 14302 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:43.699476 14276 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:43.804531 14311 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:43.828090  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 414618us
W20260709 13:05:43.828977  9826 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.732s (SENT) [suppressed 11 similar messages]
W20260709 13:05:43.830643  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:43.831113  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.688715 14260 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:43.808516 14052 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:43.833033  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.753592 13683 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:43.840610 14412 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:43.841481  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.822166 14416 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:43.846948 14420 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:05:43.847602 14411 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:05:43.860896 14302 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:43.862131 14185 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:43.865674 14369 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:43.870671  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:43.869530 14276 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:43.882606 14404 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:05:43.883394 14422 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:05:43.884696 14285 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:43.886708 14369 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:43.877749  9828 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.887377  9190 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:43.880872  9116 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:43.887954 14276 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:43.876044  9117 raft_consensus.cc:1217] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Deduplicated request from leader. Original: 12.11->[13.12-13.12]   Dedup: 13.12->[]
I20260709 13:05:43.888825  9117 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:05:43.898352  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:43.902093  9193 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:43.904412 14376 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:43.904138 14306 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:43.905359 14306 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:43.905754 14306 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:43.906250 14308 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:43.905752 14312 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:43.909852 14424 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.12, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:43.910988 14424 consensus_queue.cc:1243] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:05:43.923455 14309 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:43.890475 14421 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:43.924505 14421 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:43.925020 14421 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:43.890125 14302 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:43.924039 14309 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:43.933815  9196 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:43.935179  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:43.897857  9164 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:43.936707  9089 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:05:43.944120  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 329727us
I20260709 13:05:43.945701  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.947009  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.947685  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.948007  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.948495  9269 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:43.948844  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.947643  9263 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:05:43.949471  9272 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
W20260709 13:05:43.952853  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55109.9us
I20260709 13:05:43.964051  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.965739  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.967811  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:43.968252  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.968559  9263 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:43.968417  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.969756  9267 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:05:43.245623  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:43.972848  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:43.973902  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.974635  9268 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:43.975114  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.971256  9087 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:43.978147  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:43.981860  9263 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:05:43.982333  9263 pending_rounds.cc:85] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 9
I20260709 13:05:43.982601  9263 pending_rounds.cc:107] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:05:43.984879  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:44.001020  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:05:44.002151  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 47776.4us
I20260709 13:05:44.003309  9826 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:44.005137  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:43.890663 13814 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 13:05:43.987087  9263 raft_consensus.cc:2889] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:43.987881  9089 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:43.989081  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:43.998755  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.970561 14269 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:43.961946 14376 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:43.999579  9858 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:44.014802  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:44.028903  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:44.033556  9854 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:44.034812  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 45850.1us
W20260709 13:05:44.038275  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 50526us
I20260709 13:05:44.041071  9267 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:05:44.042084  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:43.981709 13980 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:05:44.013722 14200 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:44.048089 14196 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:44.048226  9018 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:44.049160 13804 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:44.049840 13804 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:44.050791 14348 consensus_meta.cc:287] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.609s	user 0.006s	sys 0.004s
I20260709 13:05:44.051367  9121 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:44.054001 14399 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:44.054466 14399 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:44.054848 14399 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.055949 14375 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:05:44.057186 14375 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:44.057587 14375 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:43.986158 14362 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:43.986068 14417 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:44.058394 14375 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:44.058801 14414 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:44.059208 14417 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:44.059590 14417 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:44.021752 14254 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:05:44.064927 14311 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.066447  9270 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:43.894464 14255 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.051118  9018 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:44.021087 14404 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:05:44.020179 14349 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:05:43.985813 13824 consensus_queue.cc:1059] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 13:05:43.983312 13898 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:44.098646  9854 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:44.045943  9268 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:05:44.032385 14269 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:44.027369 14185 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:43.999711 14371 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:05:44.067309 14230 consensus_meta.cc:287] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.691s	user 0.003s	sys 0.007s
I20260709 13:05:44.037779 14355 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.038636 14376 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.050951 13804 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:44.124176  9195 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:05:44.069365 14424 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:44.121448  9858 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:44.095978 14375 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:44.096221 14404 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:44.067137 14311 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:44.163722  9018 catalog_manager.cc:5697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 6 to 8, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 8 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:05:44.124388  9017 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42488 (request call id 240) took 48120 ms (client timeout 15000 ms). Trace:
I20260709 13:05:44.103682 13898 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.167464  9266 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:44.169488  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:44.101174 14296 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:05:44.170928  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:44.115825 14269 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:44.174075  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:44.180126  9265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.012s	sys 0.024s
W20260709 13:05:44.200356  9265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.012s	sys 0.025s
I20260709 13:05:44.187103  9018 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 4 to 7, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 7 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:44.183403  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:44.120398 14425 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:44.099463 14196 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:05:44.198932  9825 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:44.022120 14309 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.188796  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:44.201792  9855 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 18: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:44.213066  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 183752us
I20260709 13:05:44.164480 14376 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:44.187775 14404 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:44.213973  9122 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 18: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:44.220408 13601 raft_consensus.cc:3055] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:05:44.222121 13601 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:44.196416 13898 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:44.192788 14339 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:44.226874  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 174234us
I20260709 13:05:44.232602 13601 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:44.239192  9266 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 2223) took 1668 ms (client timeout 2022 ms). Trace:
I20260709 13:05:44.205770 14425 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:44.189294 14230 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260709 13:05:44.239666  9266 rpcz_store.cc:269] 0709 13:05:42.570618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.571078 (+   460us) service_pool.cc:224] Handling call
0709 13:05:44.239024 (+1667946us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54000ea6d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:44.239132 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3591959424}
W20260709 13:05:44.243851  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 285405us
I20260709 13:05:44.167907 14355 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:44.219605 13814 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:44.224169 14276 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:05:44.226084 14183 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 16 LEARNER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:44.263777  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 181732us
I20260709 13:05:44.265117  9238 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:44.173118 14424 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:44.244647 14416 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:44.268455  9197 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:05:44.267524  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:44.281901  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:44.282521  9854 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:44.272940 14345 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:05:44.284370 14434 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:05:44.238726 14348 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:05:44.288596  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:44.266791 14260 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:44.305649  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:44.306855  9854 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:44.299458  9122 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:44.256286 14115 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 13:05:44.187142  9017 rpcz_store.cc:269] 0709 13:04:55.991986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:56.002241 (+ 10255us) service_pool.cc:224] Handling call
0709 13:05:44.112145 (+48109904us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:38.364447 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:38.365381 (+   934us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:38.365451 (+    70us) write_op.cc:435] Acquired schema lock
0709 13:05:38.365497 (+    46us) tablet.cc:662] Decoding operations
0709 13:05:38.366326 (+   829us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:38.366459 (+   133us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:38.366504 (+    45us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:05:38.367138 (+   634us) tablet.cc:701] Row locks acquired
0709 13:05:38.367211 (+    73us) write_op.cc:260] PREPARE: finished
0709 13:05:38.367443 (+   232us) write_op.cc:270] Start()
0709 13:05:38.367651 (+   208us) write_op.cc:276] Timestamp: P: 1783602338367360 usec, L: 0
0709 13:05:38.367706 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 13:05:38.434942 (+ 67236us) log.cc:844] Serialized 1293 byte log entry
0709 13:05:38.762049 (+327107us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b540001a4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566dfe28 00007f765a11a581 00007f765a119d58 00007f765a11e9f1 00007f765a11e9a9 00007f765a11e939 00007f765a11e901 00007f765a11db2d 000000000038f354 000000000038f199
0709 13:05:38.949006 (+186957us) op_driver.cc:464] REPLICATION: finished
0709 13:05:39.075845 (+126839us) write_op.cc:301] APPLY: starting
0709 13:05:39.075956 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:39.078778 (+  2822us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:39.078816 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:39.088841 (+ 10025us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:39.089127 (+   286us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:05:39.089204 (+    77us) write_op.cc:312] APPLY: finished
0709 13:05:39.089486 (+   282us) log.cc:844] Serialized 41 byte log entry
0709 13:05:39.089695 (+   209us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:39.089795 (+   100us) write_op.cc:454] Released schema lock
0709 13:05:39.089850 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:39.105539 (+ 15689us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:39.111146 (+  5607us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":206227,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":122592,"apply.run_cpu_time_us":6486,"apply.run_wall_time_us":14455,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16673,"num_ops":3,"prepare.queue_time_us":349920,"prepare.run_cpu_time_us":24111,"prepare.run_wall_time_us":403575,"raft.queue_time_us":130014,"raft.run_cpu_time_us":11023,"raft.run_wall_time_us":205814,"replication_time_us":561686,"spinlock_wait_cycles":96622976,"thread_start_us":681108,"threads_started":4,"wal-append.queue_time_us":276926,"wal-append.run_cpu_time_us":23620,"wal-append.run_wall_time_us":1378499}]]}
I20260709 13:05:44.318403  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:44.319705  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 52263.1us
I20260709 13:05:44.321285  9854 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:44.325744  9854 raft_consensus.cc:3055] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:05:44.326244  9854 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:44.296442 14276 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.330840  9826 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:44.332037  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:44.309285 14402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.010s	sys 0.017s
I20260709 13:05:44.239202 14339 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.327137  9854 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.319410 13894 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.338569 14230 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:05:44.356581 13894 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:44.358982 13894 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:44.319512 14434 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:44.308883 14265 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:05:44.333523 14255 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:44.377709  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:44.339035 14185 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:44.338809 14416 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:44.353897 14402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.011s	sys 0.018s
W20260709 13:05:44.400331  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 139193us
I20260709 13:05:44.402588  9196 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:05:44.307844 14435 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:05:44.319980 14447 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:05:44.328516 14345 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:05:44.364195 14230 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:44.392165 14115 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.340062 14276 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.294472 14425 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:44.365011 14428 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.216s
I20260709 13:05:44.411778 14416 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:44.431969 14447 consensus_queue.cc:1243] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:05:44.400460 13814 consensus_queue.cc:1059] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:05:44.424896 14434 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:44.448314 14312 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:44.455220 14416 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:44.441357 14230 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.484489 14425 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:44.444491 14115 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:44.497759 14052 consensus_meta.cc:287] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.657s	user 0.011s	sys 0.000s
I20260709 13:05:44.510566  9196 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:44.483947 14355 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:44.511548  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:44.494664 14345 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:44.520205  9193 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:44.512048 14413 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:05:44.534502 14444 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 13:05:44.539870 14436 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
W20260709 13:05:44.540215 14437 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:05:44.540527 14458 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:44.541457 14453 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:05:44.543500  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2955) took 1003 ms. Trace:
I20260709 13:05:44.543879 14449 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:44.560331 14399 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
I20260709 13:05:44.575984 14449 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.576738 14399 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.578590 14449 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:44.544894 14443 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 13:05:44.580518  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 67851.1us
I20260709 13:05:44.582180  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:44.548841 14417 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.584051  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:44.511377 14445 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:05:44.508247  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 175235us
W20260709 13:05:44.559906 14441 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
W20260709 13:05:44.560395 14442 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:05:44.644390  9862 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:44.647540 14466 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:44.648463 14463 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:05:44.648838 14466 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:44.649318 14466 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:44.650477  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2705) took 1198 ms. Trace:
I20260709 13:05:44.650280 14466 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.650868  9114 rpcz_store.cc:276] 0709 13:05:43.454835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.454972 (+   137us) service_pool.cc:224] Handling call
0709 13:05:43.455254 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.522741 (+ 67487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.522772 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.523205 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.868264 (+345059us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b1800035620. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:05:44.125442 (+257178us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:44.125495 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.125597 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.650163 (+524566us) raft_consensus.cc:1685] finished
0709 13:05:44.650185 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.650406 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86532,"raft.queue_time_us":67800,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":48804096,"thread_start_us":256979,"threads_started":1,"wal-append.queue_time_us":750254}
I20260709 13:05:44.654083 14360 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:44.654762 14309 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:44.655573 14255 consensus_queue.cc:1059] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:44.656970 14296 raft_consensus.cc:3055] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:44.657416 14296 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:44.658684  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.658353 14296 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:44.662833  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.663103  9122 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:05:44.663515  9122 pending_rounds.cc:85] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 6
I20260709 13:05:44.627990 14200 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.586196 14458 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:44.664057  9121 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 941) took 1738 ms (client timeout 1518 ms). Trace:
I20260709 13:05:44.524199 14052 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.663803  9122 pending_rounds.cc:107] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 12.7
W20260709 13:05:44.670614  9121 rpcz_store.cc:269] 0709 13:05:42.925655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:42.925867 (+   212us) service_pool.cc:224] Handling call
0709 13:05:44.663864 (+1737997us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54002d32d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:44.663966 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2586461568}
I20260709 13:05:44.676151  9122 raft_consensus.cc:2889] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:44.664398  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 80453.7us
I20260709 13:05:44.664400  9119 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:05:44.677726  9119 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 9
I20260709 13:05:44.678339  9119 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 15.10
I20260709 13:05:44.678579  9119 raft_consensus.cc:2917] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Aborting config change with OpId 15.10: Aborted: Op aborted by new leader
I20260709 13:05:44.678926  9119 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:44.673182  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:44.663537  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.585345 14417 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:44.565335 14261 consensus_meta.cc:287] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.747s	user 0.011s	sys 0.000s
I20260709 13:05:44.565645 14306 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:44.568450 14450 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.123s	user 0.002s	sys 0.000s
I20260709 13:05:44.664695 14451 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:44.569335 14454 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:05:44.569872 14424 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:44.571333 14115 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:44.523721 14345 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:44.481413 14434 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:44.584271  9198 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:05:44.498041 14312 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.586495  9861 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.574370  9265 rpcz_store.cc:276] 0709 13:05:43.539872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.540539 (+   667us) service_pool.cc:224] Handling call
0709 13:05:43.540838 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:44.200720 (+659882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.200750 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:44.201306 (+   556us) log.cc:844] Serialized 52 byte log entry
0709 13:05:44.365552 (+164246us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:44.365592 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.365654 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.543282 (+177628us) raft_consensus.cc:1685] finished
0709 13:05:44.543303 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.543440 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16762,"raft.queue_time_us":54523,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":10687,"thread_start_us":777176,"threads_started":2,"wal-append.queue_time_us":281344}
I20260709 13:05:44.717412  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:44.718164  9265 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:44.595504  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.586092  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:44.607446  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:44.623596  9826 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:44.627028  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:44.720824  9115 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
W20260709 13:05:44.627367  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.634157  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:44.677659  9272 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.017s	sys 0.008s
W20260709 13:05:44.723053  9272 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.017s	sys 0.008s
I20260709 13:05:44.560532 14416 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:44.668226 14430 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.766s	user 0.003s	sys 0.001s
W20260709 13:05:44.663728 14440 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
W20260709 13:05:44.568658 14421 consensus_meta.cc:287] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.004s
I20260709 13:05:44.704527 14306 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.668124 14458 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:44.668241 14276 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:44.675501 14466 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:44.703673  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 166170us
I20260709 13:05:44.703925 14230 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:44.715142  9164 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:05:44.725454  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 101971us
I20260709 13:05:44.763216  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:44.764168  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:44.764633  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:44.665167 14200 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.700946 14451 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:44.694419 14261 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.738461 14358 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.739217 14371 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.740257  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:44.796319  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 56160us
I20260709 13:05:44.740365 14375 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:44.746251 14416 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:44.747713 14421 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:44.808056  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 92998.9us
I20260709 13:05:44.809051  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:44.809755  9194 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:44.753432  9854 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:44.759923  9198 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:05:44.707360 14355 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:44.767534  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:44.772563  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:44.716645 14399 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:44.781441  9265 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:44.699136 13898 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.731526 14417 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:44.792281  9859 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:44.793613  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:44.836522  9855 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:44.795178  9862 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 13:05:44.799171  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:44.796334 14445 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:44.774227 14285 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:44.768982 14432 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:44.803352  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:44.857344  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 93306.5us
W20260709 13:05:44.788386 14453 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:05:44.805967  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:44.815430  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49280.1us
I20260709 13:05:44.783385 14312 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:44.779361 14451 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.822057  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:44.922806  9118 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:44.839020  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:44.803186 14404 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.003s
I20260709 13:05:44.702203 14376 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.796554 14276 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:44.848551  9857 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:44.851055  9120 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:44.855448  9854 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:05:45.129199  9854 pending_rounds.cc:85] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 8
I20260709 13:05:45.040511 14285 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:44.805672  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1455) took 1534 ms. Trace:
I20260709 13:05:44.834264 14447 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:44.917232  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 263043us
I20260709 13:05:45.153559  9856 rpcz_store.cc:276] 0709 13:05:43.271552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.272032 (+   480us) service_pool.cc:224] Handling call
0709 13:05:43.272519 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.273022 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:43.273062 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:43.273675 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 13:05:43.648901 (+375226us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b18004926a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1 00007f765669fa0f
0709 13:05:43.757931 (+109030us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:43.757982 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:43.758060 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.667507 (+909447us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:44.805480 (+137973us) raft_consensus.cc:1685] finished
0709 13:05:44.805500 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:44.805617 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":34899,"spinlock_wait_cycles":16159744,"thread_start_us":108817,"threads_started":1,"wal-append.queue_time_us":559139}
I20260709 13:05:44.833285 14416 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:44.926031  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:45.044526  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:45.043462  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 170025us
I20260709 13:05:45.045225  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:45.043068  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:45.120886  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.166464  9196 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:44.801963 14371 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.823633 14345 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.000712  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:44.916749 14414 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.124270  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2956) took 1573 ms. Trace:
I20260709 13:05:45.127425  9861 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:45.044782  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:45.056680  9117 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:45.001778 13170 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:44.799716 14358 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.003378  9853 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:05:45.129473  9854 pending_rounds.cc:107] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:05:44.807438 14362 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:45.132045  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:45.150637 14052 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.233664  9264 raft_consensus.cc:2368] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12.
I20260709 13:05:45.234805  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:45.235576  9264 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:05:45.236279 13601 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:45.236934  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:45.119462 14441 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 13:05:44.856993 14445 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:44.834146 14355 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:45.021535 14404 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.184263  9859 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:45.183497 14468 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.425s	user 0.000s	sys 0.001s
I20260709 13:05:45.130034 14475 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:45.256028  9123 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:45.230778  9854 raft_consensus.cc:2889] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.183199 14424 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:45.197850  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.241997 14345 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:45.168648 14339 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.297435 14358 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.254166 14230 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.299091 14230 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.301215 14355 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:05:45.301327 14230 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.243975 13170 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.302074 14261 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:45.288252 14472 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:05:45.302968 14404 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.303277  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2711) took 1840 ms. Trace:
I20260709 13:05:45.303622  9119 rpcz_store.cc:276] 0709 13:05:43.462473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.484862 (+ 22389us) service_pool.cc:224] Handling call
0709 13:05:44.664118 (+1179256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.679438 (+ 15320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.303079 (+623641us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002d32d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:45.303221 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2711496576}
I20260709 13:05:45.304205  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2251) took 1263 ms. Trace:
I20260709 13:05:45.304780  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2247) took 1265 ms. Trace:
I20260709 13:05:45.240866 14414 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.185406 14447 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:45.310186  9196 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:45.295432 14481 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:45.292263  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2309) took 1766 ms. Trace:
I20260709 13:05:45.313833 14481 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:45.314378 14481 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:45.313901  9194 rpcz_store.cc:276] 0709 13:05:43.525822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.536850 (+ 11028us) service_pool.cc:224] Handling call
0709 13:05:43.537131 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.810170 (+1273039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.291984 (+481814us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400068bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:45.292199 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2926412544}
I20260709 13:05:45.292500 14474 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:45.293300 14480 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.457s	user 0.001s	sys 0.000s
W20260709 13:05:45.294097 14477 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
W20260709 13:05:45.295059 14487 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:05:45.298138 14424 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.040345 14276 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.360209  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:45.304653  9270 rpcz_store.cc:276] 0709 13:05:44.040918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.067454 (+ 26536us) service_pool.cc:224] Handling call
0709 13:05:44.084085 (+ 16631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.362266 (+278181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.362296 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.398000 (+ 35704us) log.cc:844] Serialized 26 byte log entry
0709 13:05:44.700793 (+302793us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:44.744036 (+ 43243us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.759083 (+ 15047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.759192 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.303991 (+544799us) raft_consensus.cc:1685] finished
0709 13:05:45.304012 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.304146 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24322,"prepare.queue_time_us":43283,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":127037,"raft.run_cpu_time_us":497,"raft.run_wall_time_us":21722,"thread_start_us":544408,"threads_started":2,"wal-append.queue_time_us":425940}
I20260709 13:05:45.174436 14449 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:45.232196  9271 rpcz_store.cc:276] 0709 13:05:43.551289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:43.551442 (+   153us) service_pool.cc:224] Handling call
0709 13:05:43.551769 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:43.589008 (+ 37239us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:43.631078 (+ 42070us) log.cc:844] Serialized 14 byte log entry
0709 13:05:43.853919 (+222841us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b18003531a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:05:44.445871 (+591952us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.484179 (+ 38308us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.484328 (+   149us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.636533 (+152205us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b18003531a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:44.760860 (+124327us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b2c00112948. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:05:44.761239 (+   379us) log.cc:844] Serialized 26 byte log entry
0709 13:05:44.761423 (+   184us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:44.761472 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.761549 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:44.873609 (+112060us) raft_consensus.cc:1685] finished
0709 13:05:44.873637 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.124194 (+250557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38901,"prepare.queue_time_us":355063,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":5666,"raft.queue_time_us":193196,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":32860,"spinlock_wait_cycles":32944000,"thread_start_us":626488,"threads_started":4,"wal-append.queue_time_us":612951}
W20260709 13:05:45.372005 14478 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.351s	user 0.003s	sys 0.000s
I20260709 13:05:45.309372  9854 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:45.304785 14376 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.261283 14413 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.312786 14447 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.326633  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:45.389472  9116 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:45.303629 13170 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.305222  9272 rpcz_store.cc:276] 0709 13:05:44.039727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.043846 (+  4119us) service_pool.cc:224] Handling call
0709 13:05:44.090547 (+ 46701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.723503 (+632956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.723547 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.729781 (+  6234us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.074192 (+344411us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:45.165207 (+ 91015us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.170848 (+  5641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.171008 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.304586 (+133578us) raft_consensus.cc:1685] finished
0709 13:05:45.304606 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.304724 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64792,"prepare.queue_time_us":90726,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":135264,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":65817,"thread_start_us":904093,"threads_started":2,"wal-append.queue_time_us":514144}
W20260709 13:05:45.301498 14425 consensus_meta.cc:287] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.787s	user 0.008s	sys 0.000s
I20260709 13:05:45.278509 14421 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:45.348692  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.157753 14115 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:45.304693 14371 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:45.358934  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.626102  9268 raft_consensus.cc:2410] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:05:45.628750  9115 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:05:45.362016  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:45.311519 14434 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:45.366726 14309 consensus_meta.cc:287] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.711s	user 0.009s	sys 0.001s
I20260709 13:05:45.309114 14473 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:45.287786 14475 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:45.349146 14200 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.357834 14474 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.701825 14479 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:45.702152 14474 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.702747 14479 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.712105 14473 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:45.714299 14482 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.329s	user 0.000s	sys 0.000s
I20260709 13:05:45.713709 14473 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.716609  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2317) took 1209 ms. Trace:
I20260709 13:05:45.717080  9197 rpcz_store.cc:276] 0709 13:05:44.507604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.507766 (+   162us) service_pool.cc:224] Handling call
0709 13:05:44.520466 (+ 12700us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:44.527825 (+  7359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.527873 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:44.552218 (+ 24345us) log.cc:844] Serialized 52 byte log entry
0709 13:05:44.839921 (+287703us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:44.860389 (+ 20468us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.999427 (+139038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.999511 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.716305 (+716794us) raft_consensus.cc:1685] finished
0709 13:05:45.716333 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.716535 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56100,"prepare.queue_time_us":20310,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":152036,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":165,"thread_start_us":211585,"threads_started":1,"wal-append.queue_time_us":734029}
I20260709 13:05:45.383899 14449 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.365295 14445 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:45.365563 14230 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:45.348158 14375 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:45.383970  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:45.363436 14269 consensus_meta.cc:287] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.139s	user 0.013s	sys 0.000s
I20260709 13:05:45.164259 14285 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.393131  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1466) took 1104 ms. Trace:
I20260709 13:05:45.398125 13170 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:45.744606 13170 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:45.246613 14476 consensus_queue.cc:1175] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "94504d29c6744f38bb06943479df9466" responder_term: 13 status { last_received { term: 11 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: [12.12-12.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:05:45.653287  9270 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:45.386309 14404 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:45.669399  9263 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:05:45.624384 14355 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.759624 14355 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.762110 14355 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.365151 14399 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:45.621596 14367 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:45.622622 14196 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:45.650573 14425 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:45.739151  9116 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:45.658597 14371 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:45.742275  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:45.748358  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2971) took 1055 ms. Trace:
W20260709 13:05:45.752345  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.743588  9857 rpcz_store.cc:276] 0709 13:05:44.288798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.288921 (+   123us) service_pool.cc:224] Handling call
0709 13:05:44.289245 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.926686 (+637441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.392792 (+466106us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b5400738f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:45.393050 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285753728}
I20260709 13:05:45.743098 14416 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:45.764803 14424 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.767926 14474 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.770268 13898 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:45.770959 14306 consensus_meta.cc:287] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.992s	user 0.003s	sys 0.007s
I20260709 13:05:45.659852 14413 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:45.692498  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:45.775981  9272 pending_rounds.cc:85] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 8
I20260709 13:05:45.824504  9272 pending_rounds.cc:107] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:05:45.824854  9272 raft_consensus.cc:2889] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:45.732371 14479 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.731516 13824 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:45.649358 14115 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:45.780432  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.049s	sys 0.123s
W20260709 13:05:45.835765  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.049s	sys 0.124s
I20260709 13:05:45.835728  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1127) took 1496 ms. Trace:
I20260709 13:05:45.737102 14434 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:45.786759 13170 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:45.778604 14302 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.827s	user 0.015s	sys 0.000s
I20260709 13:05:45.794973  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1230) took 1004 ms. Trace:
I20260709 13:05:45.796511  9270 rpcz_store.cc:276] 0709 13:05:44.693040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.652349 (+959309us) service_pool.cc:224] Handling call
0709 13:05:45.652833 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.747962 (+ 95129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.748277 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.773387 14414 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:45.743650 14230 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:45.755035 14375 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.848208  9853 rpcz_store.cc:276] 0709 13:05:44.790240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.790391 (+   151us) service_pool.cc:224] Handling call
0709 13:05:44.790751 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.231516 (+440765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.794697 (+563181us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400700154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:45.794908 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463311744}
I20260709 13:05:45.860383  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:45.793095 14399 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:45.775167 14269 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:45.837185  9266 rpcz_store.cc:276] 0709 13:05:44.339133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.339769 (+   636us) service_pool.cc:224] Handling call
0709 13:05:44.356653 (+ 16884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.465530 (+108877us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.465581 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.466080 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:05:44.692840 (+226760us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:44.710538 (+ 17698us) log.cc:844] Serialized 14 byte log entry
0709 13:05:44.710822 (+   284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.710921 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:45.371333 (+660412us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:45.831475 (+460142us) raft_consensus.cc:1685] finished
0709 13:05:45.835468 (+  3993us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.835652 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32632,"prepare.queue_time_us":17465,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":4428,"raft.queue_time_us":104315,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":19483,"spinlock_wait_cycles":29157504,"thread_start_us":299119,"threads_started":2,"wal-append.queue_time_us":322887}
I20260709 13:05:45.821413 14445 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:45.842180 14345 consensus_meta.cc:287] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.001s
I20260709 13:05:45.778887 14404 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.791177 14367 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.832543 14309 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.792619 14196 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:45.880201  9270 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:45.852528 14493 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.853027 13898 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:45.795666 14425 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.823683 14371 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:45.892035  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:45.902928  9266 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:45.833660 14474 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.895155  9114 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:45.843362  9271 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:45.844997 13824 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.857296 14302 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.908004  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2972) took 1214 ms. Trace:
I20260709 13:05:45.910269  9271 rpcz_store.cc:276] 0709 13:05:44.693289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.834524 (+1141235us) service_pool.cc:224] Handling call
0709 13:05:45.842927 (+  8403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.906414 (+ 63487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.907935 (+  1521us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:05:45.901757 14451 consensus_meta.cc:287] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.847s	user 0.011s	sys 0.000s
I20260709 13:05:45.859793 14434 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.887535 14445 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.912319  9162 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:45.888505 13980 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:05:45.917431  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2973) took 1207 ms. Trace:
I20260709 13:05:45.897647 14493 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:45.919149  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:45.925427  9271 rpcz_store.cc:276] 0709 13:05:44.708082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.915042 (+1206960us) service_pool.cc:224] Handling call
0709 13:05:45.915342 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.915510 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.915532 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.915568 (+    36us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:45.915595 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.915646 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.915772 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.895787 14309 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:45.926949  9266 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:45.924067  9270 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:45.910809 14276 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.928473 13170 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:45.903685 13898 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.913092 14475 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:45.894645 14416 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.929831 13170 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:45.921275 14496 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:45.919691 14449 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.938177  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2974) took 1178 ms. Trace:
I20260709 13:05:45.938493  9266 rpcz_store.cc:276] 0709 13:05:44.760028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.929695 (+1169667us) service_pool.cc:224] Handling call
0709 13:05:45.937827 (+  8132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.937956 (+   129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.937973 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.937999 (+    26us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:45.938019 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.938056 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.938128 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.915181 14451 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:45.946746 13170 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.934844 14285 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.940461 14496 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:45.951378 14466 consensus_meta.cc:287] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.179s	user 0.006s	sys 0.005s
W20260709 13:05:45.953843 14481 consensus_meta.cc:287] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.639s	user 0.003s	sys 0.008s
I20260709 13:05:45.949278  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:45.926060 14494 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:05:45.921041 14399 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.959996 14496 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:45.970676 14269 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.962795 14481 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:45.972036 14367 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.973276 14371 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.963459  9267 raft_consensus.cc:2410] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:05:45.975821  9267 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2963) took 1563 ms (client timeout 1634 ms). Trace:
I20260709 13:05:45.976048 14481 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.976321  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2376) took 1420 ms. Trace:
I20260709 13:05:45.964799 14493 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.976774 14445 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.977196 14493 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:45.977744 14493 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:45.978220 14449 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:45.969395 14425 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.987218  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:45.988245 14404 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:45.988876  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:45.989114 14474 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:45.976199  9267 rpcz_store.cc:269] 0709 13:05:44.412286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.607163 (+194877us) service_pool.cc:224] Handling call
0709 13:05:45.975613 (+1368450us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54000fb4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:45.975760 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859895040}
I20260709 13:05:45.980556  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:45.990269  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:45.989197 13898 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:45.957533 14230 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:45.976807  9193 rpcz_store.cc:276] 0709 13:05:44.556082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.556212 (+   130us) service_pool.cc:224] Handling call
0709 13:05:44.556490 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:44.668365 (+111875us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:44.668681 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.162586 (+493905us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.386625 (+224039us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.386683 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:45.386736 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:45.976124 (+589388us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400398354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:45.976165 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:45.976268 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":405515,"prepare.queue_time_us":6580,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":113,"raft.queue_time_us":405506,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":71868,"spinlock_wait_cycles":256750848,"thread_start_us":326711,"threads_started":2,"wal-append.queue_time_us":377399}
I20260709 13:05:46.000240  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:46.000727  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.004333  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.005167  9267 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:05:46.006153  9196 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:46.006165  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.001300  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.009876  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2975) took 1249 ms. Trace:
I20260709 13:05:46.010335  9267 rpcz_store.cc:276] 0709 13:05:44.760380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.008895 (+1248515us) service_pool.cc:224] Handling call
0709 13:05:46.009235 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.009458 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.009501 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.009562 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:46.009609 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.009692 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.009815 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:45.960327 13824 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.013247  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:46.014627  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.015539  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:45.970482 14302 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:46.018190  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.021329  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.023473  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2976) took 1262 ms. Trace:
I20260709 13:05:45.999318 14493 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.025491  9267 rpcz_store.cc:276] 0709 13:05:44.760678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.011412 (+1250734us) service_pool.cc:224] Handling call
0709 13:05:46.011831 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.023045 (+ 11214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.023084 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.023125 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:46.023158 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.023237 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.023403 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.037482  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2257) took 1190 ms. Trace:
I20260709 13:05:46.038056  9271 rpcz_store.cc:276] 0709 13:05:44.847460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.027237 (+1179777us) service_pool.cc:224] Handling call
0709 13:05:46.036780 (+  9543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.037022 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.037059 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.037113 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:46.037174 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.037273 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.037410 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.045820  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2977) took 1203 ms. Trace:
I20260709 13:05:46.035022 14434 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.048588  9267 rpcz_store.cc:276] 0709 13:05:44.842432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.027226 (+1184794us) service_pool.cc:224] Handling call
0709 13:05:46.027513 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.027676 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:46.029144 (+  1468us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.044951 (+ 15807us) log.cc:844] Serialized 14 byte log entry
0709 13:05:46.045372 (+   421us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.045438 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:46.045497 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.045587 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.045745 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2383,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":91,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":854,"thread_start_us":10910,"threads_started":1}
I20260709 13:05:46.050757  9271 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:46.052211  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2258) took 1204 ms. Trace:
I20260709 13:05:46.056061  9271 rpcz_store.cc:276] 0709 13:05:44.847783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.039321 (+1191538us) service_pool.cc:224] Handling call
0709 13:05:46.047515 (+  8194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.051492 (+  3977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.052160 (+   668us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.057665  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1129) took 1192 ms. Trace:
I20260709 13:05:46.057992  9271 rpcz_store.cc:276] 0709 13:05:44.865347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.056862 (+1191515us) service_pool.cc:224] Handling call
0709 13:05:46.057184 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.057357 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.057383 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.057424 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:46.057457 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.057508 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.057614 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.059877 14493 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.060492 14493 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:46.062556 14493 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.052395  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2259) took 1204 ms. Trace:
I20260709 13:05:46.063179  9267 rpcz_store.cc:276] 0709 13:05:44.848042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.051586 (+1203544us) service_pool.cc:224] Handling call
0709 13:05:46.051950 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.052122 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.052154 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:46.052188 (+    34us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:05:46.052218 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.052268 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.052352 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:05:46.066957 14493 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:46.068430  9861 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 1469) took 1733 ms (client timeout 1785 ms). Trace:
W20260709 13:05:46.068878  9861 rpcz_store.cc:269] 0709 13:05:44.335118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.581862 (+246744us) service_pool.cc:224] Handling call
0709 13:05:46.068216 (+1486354us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b54005ee554. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:46.068347 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8289536}
I20260709 13:05:46.067519 14493 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.072332  9267 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:05:46.073004 14506 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:05:46.041780 14375 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.065003  9117 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 951) took 1868 ms. Trace:
I20260709 13:05:46.087965 14269 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.088570 14269 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:46.086052  9117 rpcz_store.cc:276] 0709 13:05:44.197102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.197987 (+   885us) service_pool.cc:224] Handling call
0709 13:05:44.198357 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.252456 (+1054099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.064752 (+812296us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b540023bed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:46.064942 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973363456}
I20260709 13:05:46.097319 14185 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:46.097898 14185 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.099339 14296 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:46.100119 14185 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:46.108932 14493 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:46.109781  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72931.6us
I20260709 13:05:46.094273 14375 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:46.119057 14485 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.813s	user 0.000s	sys 0.013s
I20260709 13:05:46.122189  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2969) took 1430 ms. Trace:
I20260709 13:05:46.124982  9114 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2627) took 1322 ms. Trace:
I20260709 13:05:46.129618  9114 rpcz_store.cc:276] 0709 13:05:44.802948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.803122 (+   174us) service_pool.cc:224] Handling call
0709 13:05:46.124819 (+1321697us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002f67d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:46.124925 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2385886592}
I20260709 13:05:46.128010  9264 rpcz_store.cc:276] 0709 13:05:44.691705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.251408 (+559703us) service_pool.cc:224] Handling call
0709 13:05:45.252072 (+   664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.700898 (+448826us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:45.701205 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 13:05:45.896643 (+195438us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:45.896692 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:45.896730 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.121967 (+225237us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54000faad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:46.122015 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.122134 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":564,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1566,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3222,"spinlock_wait_cycles":1031614848,"thread_start_us":188932,"threads_started":1,"wal-append.queue_time_us":214837}
I20260709 13:05:46.133842  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.134466  9264 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:46.135257  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.111639 14434 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:46.136754  9264 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:05:46.188623  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.190096  9264 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:46.191473  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:46.192507  9264 raft_consensus.cc:2368] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:46.193516  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:46.194039  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 79296.6us
I20260709 13:05:46.206663  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.207553  9264 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:46.208657  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.132771 14269 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.209722  9264 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
W20260709 13:05:46.212592  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.188073 14374 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.277338  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.148993 14493 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.148010 14451 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:46.243508 14185 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.245208  9120 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2629) took 1430 ms. Trace:
I20260709 13:05:46.188622 14376 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.195866 14399 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:46.189699 14255 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.226958 14448 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:46.283344  9120 rpcz_store.cc:276] 0709 13:05:44.814776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.814891 (+   115us) service_pool.cc:224] Handling call
0709 13:05:44.821140 (+  6249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.152429 (+331289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.244872 (+1092443us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b5400255b54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:46.245129 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59317120}
I20260709 13:05:46.284569  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1470) took 1869 ms. Trace:
I20260709 13:05:46.290050  9264 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:46.288897  9858 rpcz_store.cc:276] 0709 13:05:44.415565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.628208 (+212643us) service_pool.cc:224] Handling call
0709 13:05:44.628564 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:44.670489 (+ 41925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.670518 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:44.676368 (+  5850us) log.cc:844] Serialized 52 byte log entry
0709 13:05:44.868576 (+192208us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:44.868636 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:44.868733 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.118490 (+1249757us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:46.263946 (+145456us) raft_consensus.cc:1685] finished
0709 13:05:46.263978 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.284479 (+ 20501us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41517,"raft.queue_time_us":42044,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":4281,"spinlock_wait_cycles":7900416,"thread_start_us":161194,"threads_started":1,"wal-append.queue_time_us":472729}
I20260709 13:05:46.135934 14375 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.193393 14434 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.295279 14296 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.286478 14185 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.295676 14255 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.372838 14404 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.287976 14493 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:46.368167 14469 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.295809 14399 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.292690 14374 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.297644 14448 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.371865 14269 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:46.295770 14376 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.503423 14185 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.422286 14404 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.513542 14255 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.514204 14255 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.514750 14255 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.424595 14200 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:46.515789 14200 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.521412 14200 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.524636 14200 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.522665 14434 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.523140 14404 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.499070 14269 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:46.540280  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.541949  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:46.542160  9120 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
W20260709 13:05:46.532351 14512 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
I20260709 13:05:46.543694  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:46.539847 14404 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.544714  9116 raft_consensus.cc:2368] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12.
W20260709 13:05:46.545203 14517 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:05:46.545039 14404 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.546429  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2260) took 1376 ms. Trace:
I20260709 13:05:46.547197 14404 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.547978 14404 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.546802  9271 rpcz_store.cc:276] 0709 13:05:45.169915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.058881 (+888966us) service_pool.cc:224] Handling call
0709 13:05:46.059207 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.059550 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.059580 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.074976 (+ 15396us) log.cc:844] Serialized 26 byte log entry
0709 13:05:46.219388 (+144412us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:46.219428 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.219499 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.546229 (+326730us) raft_consensus.cc:1685] finished
0709 13:05:46.546248 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.546374 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16543,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":7502,"thread_start_us":128465,"threads_started":1,"wal-append.queue_time_us":136880}
I20260709 13:05:46.548427 14404 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.549330  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:46.549949  9116 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:46.550655 14404 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:46.535802 14200 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.532495 14302 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.480578 14469 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:46.560026  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:46.561717  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:46.563514  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 151151us
I20260709 13:05:46.557663  9116 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:46.567140  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.569471  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.570117  9271 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:46.571041  9267 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:46.571609  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.572360  9116 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:46.573208  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.573788  9271 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:46.594533  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:46.595857  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.596531  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.597251  9120 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:46.598948  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.600162  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.600841  9115 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
W20260709 13:05:46.603333  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:46.605917  9117 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:46.383821 14296 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:46.627992  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.562135 14302 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.663167  9116 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:46.666378  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.668510 13683 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:46.669054 13683 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.670081 14302 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.579180 14519 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.676219  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:46.676883  9271 raft_consensus.cc:2376] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12. Re-sending same reply.
I20260709 13:05:46.665321 14399 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.682610  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.683246  9116 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:46.684152  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.685000  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.686056  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.686628  9123 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:05:46.687427  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.688239  9119 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:46.692600  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.693122  9123 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:46.695883  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.697022  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.697081  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:46.701893  9116 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:46.702728  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.703904  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.705130  9119 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:46.707954  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:46.708494  9119 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:46.709919  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.710610  9267 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:46.717453  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:46.718079  9114 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:46.716001  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.714880  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.720134  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.701686 14493 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.678408 14519 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:46.723156  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.723830  9123 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:46.719513  9120 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:46.706049 14448 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.725633  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:46.726013  9116 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:46.703696 14434 consensus_queue.cc:1059] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:05:46.743196  9119 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:46.749996  9114 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:46.753912  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.758152  9115 raft_consensus.cc:2410] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:05:46.744750 14375 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.776728 14413 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:46.772289 14496 consensus_meta.cc:287] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.801s	user 0.010s	sys 0.002s
I20260709 13:05:46.770661 14469 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.772509 14374 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.824360  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.793318 14376 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.831175 14296 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.791249 14496 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.830745 14302 consensus_queue.cc:1059] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:05:46.838042 14496 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.847930  9825 leader_election.cc:400] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12.
I20260709 13:05:46.848480  9825 leader_election.cc:403] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:05:46.849372 14185 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12.
I20260709 13:05:46.849841  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:46.851023 14185 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:46.824023 14462 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.851432  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:46.863189  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:46.825747 14413 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.828900 14399 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.866914 14413 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.866535 14462 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.869184 14196 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.870025 14462 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:46.870296  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.864301  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.873728  9271 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:46.865072 14185 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:46.866284 14414 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:46.880519  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.881145  9861 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:46.878576  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.882289  9858 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:46.878443 14185 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.884882 14185 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:46.886305 14200 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.886658  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.887279  9858 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:46.888128 14519 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.871162 14399 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.889633  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.890398  9861 raft_consensus.cc:2410] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:46.890650 14399 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.891261  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2254) took 2849 ms. Trace:
I20260709 13:05:46.892211 14413 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:46.891824  9263 rpcz_store.cc:276] 0709 13:05:44.041597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.083927 (+ 42330us) service_pool.cc:224] Handling call
0709 13:05:44.084216 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:45.754740 (+1670524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.890965 (+1136225us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54003fcfd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:46.891187 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3644572032}
I20260709 13:05:46.892081  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.894156  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2967) took 2479 ms. Trace:
I20260709 13:05:46.894685  9265 rpcz_store.cc:276] 0709 13:05:44.415096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.839691 (+424595us) service_pool.cc:224] Handling call
0709 13:05:44.843831 (+  4140us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:45.306717 (+462886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.306751 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:45.307386 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.777886 (+470500us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:05:45.777968 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.893667 (+1115699us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5400302ad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:46.893740 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.893805 (+    65us) raft_consensus.cc:1685] finished
0709 13:05:46.893826 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.894085 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18093,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":11826,"spinlock_wait_cycles":1062922112,"thread_start_us":469796,"threads_started":1,"wal-append.queue_time_us":592354}
I20260709 13:05:46.896641  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.897231  9859 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 7.
I20260709 13:05:46.898607 14115 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:46.903172  9268 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 2968) took 2479 ms (client timeout 2776 ms). Trace:
W20260709 13:05:46.903616  9268 rpcz_store.cc:269] 0709 13:05:44.423851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.633890 (+210039us) service_pool.cc:224] Handling call
0709 13:05:46.902942 (+2269052us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b54002a3354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:46.903087 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076867328}
I20260709 13:05:46.906708  9825 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:46.907431 14413 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:46.908963  9825 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:46.909724 14302 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:46.910195 14302 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.910526  9162 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:46.911887 14462 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:46.912416 14462 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:46.913246 14404 raft_consensus.cc:670] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } [suppressed 2 similar messages]
I20260709 13:05:46.913874  9269 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:05:46.914441  9269 pending_rounds.cc:85] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 11
I20260709 13:05:46.914608 14462 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:46.914714  9269 pending_rounds.cc:107] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:05:46.915213  9269 raft_consensus.cc:2889] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:46.915269 14462 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:46.915999  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:46.916265  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.916415  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.917176  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.917762  9853 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:46.920420  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2970) took 2227 ms. Trace:
I20260709 13:05:46.921660  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.921828 14404 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.920799  9272 rpcz_store.cc:276] 0709 13:05:44.692798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:45.621867 (+929069us) service_pool.cc:224] Handling call
0709 13:05:45.622255 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:45.825677 (+203422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:45.825721 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:45.826420 (+   699us) log.cc:844] Serialized 52 byte log entry
0709 13:05:45.887417 (+ 60997us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:45.887467 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:46.920060 (+1032593us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540032e954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:46.920109 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:46.920164 (+    55us) raft_consensus.cc:1685] finished
0709 13:05:46.920177 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:46.920367 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":357,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":11794,"spinlock_wait_cycles":352752000,"thread_start_us":32195,"threads_started":1,"wal-append.queue_time_us":93203}
I20260709 13:05:46.922350 14404 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.922881  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.892974 14414 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:46.924474 14267 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:46.924929 14267 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:46.924587 14414 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.925212  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:46.926127  9267 raft_consensus.cc:2410] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:46.926184  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.927073 13683 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.925318 14267 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:46.927397  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.927961 14399 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:46.928071  9856 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:46.928566 14399 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.929246  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:46.943209  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.943689  9267 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:46.929481  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.945422  9861 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:46.945822  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.946532  9193 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 16: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:46.947225  9195 raft_consensus.cc:1240] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:46.948542  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.921072  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:46.931844  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.950160  9857 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:05:46.950377  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.951103  9856 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:46.951287  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.951812  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.952030  9857 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:46.932698  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:46.931186 14255 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.937985 14404 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:46.957281  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 265016us
I20260709 13:05:46.942049 14446 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:05:46.942517 13683 raft_consensus.cc:1064] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:05:46.943889  9197 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
W20260709 13:05:46.961870  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52970.1us
I20260709 13:05:46.953733  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.953133  9853 raft_consensus.cc:2376] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12. Re-sending same reply.
I20260709 13:05:46.965005  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.957424  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:46.966383  9267 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:46.966853 13478 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:46.938493 14269 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:46.958945  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.966569  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.973114  9190 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:46.973443  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.973749  9855 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:46.968087 13478 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:46.944914 14462 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:46.974442 13478 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:46.975521  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.983192  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.987051  9264 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:05:46.987516  9264 pending_rounds.cc:85] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 11
I20260709 13:05:46.987826  9264 pending_rounds.cc:107] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:05:46.988123  9264 raft_consensus.cc:2889] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:46.985592  9087 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:46.991192  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:46.984921  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:46.993785  9857 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:46.975849 14496 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:46.997596  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.976048 14434 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:46.980055 14462 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:47.006412  9087 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:47.002869  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.998656 14473 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:05:47.012447  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:46.995322 14387 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:05:47.013056  9854 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:47.016355  9087 leader_election.cc:400] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:47.016714  9087 leader_election.cc:403] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:05:47.017187 13478 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:46.996650  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:46.980196 14522 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:46.977382 14255 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:05:47.042779  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:46.999256 14348 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:47.094942 14348 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:47.095413 14348 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:47.093808 14376 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:47.007702 14434 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:47.000568 14451 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.018658 14426 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:47.041546 14308 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:47.147202  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:47.030211 14200 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:47.031126 14269 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.040213 14196 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:47.157599  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:47.109637 14413 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:47.176069  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48509.1us
I20260709 13:05:47.033521  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:47.175957  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 179529us
I20260709 13:05:47.175720  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:47.281303  9857 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:47.283699  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:47.286006  9857 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:47.289731  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:47.290360 14404 consensus_queue.cc:1059] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:47.293603  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:47.340904  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:47.296082  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:47.343468  9854 raft_consensus.cc:3060] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:47.308537  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 106905us
I20260709 13:05:47.103042 14296 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:05:47.014657 14496 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:47.377565 14496 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:47.138979 14374 consensus_queue.cc:1059] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.188s
I20260709 13:05:47.168431 14200 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.381623  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:47.328655  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:47.356998  9238 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.360319 14302 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:47.362412 14309 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:47.363950 14115 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:47.364741 14387 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:47.287309 14413 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.365685  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1229) took 2575 ms. Trace:
I20260709 13:05:47.366070  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:47.370244 14522 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:47.248313 14426 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:47.250646 14269 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.372921 14462 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.121083 14376 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:47.392619  9268 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
W20260709 13:05:47.393371  9235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.020s	sys 0.049s
I20260709 13:05:47.392303  9860 rpcz_store.cc:276] 0709 13:05:44.789841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:44.790021 (+   180us) service_pool.cc:224] Handling call
0709 13:05:44.790482 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:44.926067 (+135585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:44.926102 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:44.926594 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:05:45.360618 (+434024us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:45.360658 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.365253 (+2004595us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b540009bad4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:47.365306 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:47.365360 (+    54us) raft_consensus.cc:1685] finished
0709 13:05:47.365373 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:47.365601 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6449,"raft.queue_time_us":133393,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":24475,"spinlock_wait_cycles":178682112,"thread_start_us":244382,"threads_started":1,"wal-append.queue_time_us":715643,"wal-append.run_cpu_time_us":1665,"wal-append.run_wall_time_us":1003130}
I20260709 13:05:47.307216 14528 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.401688  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:47.156029 14451 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:47.407375  9235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.021s	sys 0.049s
I20260709 13:05:47.408856 14426 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.399530 14302 raft_consensus.cc:3055] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:47.411075 14376 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:47.411113 14413 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:47.412076 14529 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:47.412703 14309 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.389793 14200 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:47.389729 13804 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:47.409584 14534 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:47.429127  9122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.016s	sys 0.056s
I20260709 13:05:47.351845 14473 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
W20260709 13:05:47.477900  9122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.016s	sys 0.056s
I20260709 13:05:47.430482  9858 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 15: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:47.401238 14522 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:47.433696  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:47.434223  9267 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:47.406545 14387 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.411321 14267 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.537492  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:47.462598  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.434893  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:47.411916 14399 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:47.424436 14520 raft_consensus.cc:3055] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:47.543668  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:47.544067  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 109282us
I20260709 13:05:47.544661  9860 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:47.545079  9235 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:47.440058  9854 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:47.430300 14302 raft_consensus.cc:740] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:47.384156 14434 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.478245 13804 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.425206 14528 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:47.495440 14448 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.024s	sys 0.030s
I20260709 13:05:47.538867 14522 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:47.562932  9196 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:47.580154  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:47.537220 14388 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.026s	sys 0.031s
W20260709 13:05:47.563258 14414 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.005s	sys 0.021s
I20260709 13:05:47.547273 14520 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:47.565068 14185 consensus_meta.cc:287] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.007s
W20260709 13:05:47.565718 14448 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.024s	sys 0.030s
I20260709 13:05:47.562069 13804 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:47.478629 14309 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:47.563515  9161 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.012s	sys 0.025s
I20260709 13:05:47.538456 14348 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:47.601775  9854 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
W20260709 13:05:47.593138 14388 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.027s	sys 0.032s
I20260709 13:05:47.622872  9854 pending_rounds.cc:85] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 11
I20260709 13:05:47.631476  9854 pending_rounds.cc:107] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:05:47.631848  9854 raft_consensus.cc:2889] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:47.591043 14414 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.005s	sys 0.021s
W20260709 13:05:47.622222  9161 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.012s	sys 0.025s
I20260709 13:05:47.592695 14434 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:47.636878  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 640508us
I20260709 13:05:47.479699 14529 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.601749 14432 raft_consensus.cc:3055] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:05:47.651535  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 472647us
I20260709 13:05:47.527122 14534 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:47.654343  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 620853us
I20260709 13:05:47.659662  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:47.645673  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 452672us
I20260709 13:05:47.658537  9856 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:47.638428 14375 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.662976  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:47.657305  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 481696us
W20260709 13:05:47.672371  9270 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.004s	sys 0.024s
I20260709 13:05:47.678131  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:47.672863  9270 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.004s	sys 0.024s
I20260709 13:05:47.619765 14185 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:47.688699  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.682921  9087 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:47.690599 14296 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:47.691205 14296 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:47.691692 14296 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:47.695223 14523 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:47.654953 14432 raft_consensus.cc:740] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:47.702157  9859 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.004s	sys 0.012s
W20260709 13:05:47.702662  9859 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.004s	sys 0.012s
I20260709 13:05:47.701507  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:47.706462  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:47.707573  9826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.000s	sys 0.015s
I20260709 13:05:47.707979  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:47.684840  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:47.640609 14532 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:47.717511  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:47.687106  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:05:47.708141  9826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.001s	sys 0.015s
I20260709 13:05:47.656333 14230 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:47.722970 14230 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:47.723160  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 802111us
I20260709 13:05:47.723563 14230 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:47.727577  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:47.684999  9861 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.010s	sys 0.042s
W20260709 13:05:47.729799  9861 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.010s	sys 0.043s
I20260709 13:05:47.737344  9862 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:47.700151 14374 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:47.628572 14302 consensus_queue.cc:260] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:47.743881  9263 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.001s	sys 0.014s
W20260709 13:05:47.761977  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:47.687446 14462 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:47.672809 14447 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:05:47.620424 13683 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.704250 14269 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:47.765223  9263 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.002s	sys 0.014s
I20260709 13:05:47.716477 14196 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:47.735019 14376 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:47.680871 14375 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:47.759547 14451 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:47.766342 14374 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:47.721242 14523 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:05:47.778213 14376 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:47.788645 14531 log.cc:927] Time spent T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:05:47.790755 14196 raft_consensus.cc:697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:47.796520 14548 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:47.672981 14520 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.793154 14493 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:47.796291 14404 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:47.786316 14543 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:47.780687 14399 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.930171 14451 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:47.787015 14432 consensus_queue.cc:260] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.930363 14374 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:47.944029 14376 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:47.956015 14543 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:47.933292 14546 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:47.966602 14543 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:47.973929 14519 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:47.963310 14399 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.974536 14519 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:47.976517 14399 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:47.978876 14519 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:47.980067 14537 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:05:47.980278 14519 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:47.981549 14543 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.982899 14522 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.984185 14543 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:47.985474 14522 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:47.972397 14432 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:47.953751 14548 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:47.991516 14548 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:47.973325 14520 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:47.953678 14528 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.000756 14375 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:47.973598 14546 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:48.005216 14546 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:48.024961 14546 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.026806 14546 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:48.101709  9235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.006s	sys 0.010s
W20260709 13:05:48.102087  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.102600  9235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.007s	sys 0.010s
W20260709 13:05:48.105069  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 560042us
W20260709 13:05:48.108428  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 390792us
I20260709 13:05:48.108796  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 2989) took 1201 ms. Trace:
I20260709 13:05:48.110389  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:48.114192  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:48.115720  9268 raft_consensus.cc:2368] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12.
I20260709 13:05:48.115589  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:48.116750  9272 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:48.118350  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:48.118640  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 431428us
W20260709 13:05:48.101709  9237 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.000s	sys 0.022s
W20260709 13:05:48.120252  9237 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.000s	sys 0.022s
I20260709 13:05:48.120577  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:48.122933 14576 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:48.123138  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:48.111133  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:48.123900 14576 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:48.124629 14576 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.124925  9263 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:48.127602  9266 raft_consensus.cc:2410] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:48.131246  9089 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:48.132050  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:48.132992  9265 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:48.134928  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:48.137363  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.126089  9235 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:48.109568  9270 rpcz_store.cc:276] 0709 13:05:46.907308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.907982 (+   674us) service_pool.cc:224] Handling call
0709 13:05:46.908439 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:47.103127 (+194688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:47.103157 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:47.104254 (+  1097us) log.cc:844] Serialized 26 byte log entry
0709 13:05:47.680096 (+575842us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:47.680155 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.680252 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:48.107870 (+427618us) raft_consensus.cc:1685] finished
0709 13:05:48.107962 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.108469 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6360,"raft.queue_time_us":429173,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":40113,"thread_start_us":701736,"threads_started":2,"wal-append.queue_time_us":638121}
I20260709 13:05:48.140686  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:47.773772 13824 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
W20260709 13:05:48.145849  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 418191us
I20260709 13:05:48.179061 14561 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:48.179502  9826 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:48.180464 14561 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.180665 14196 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.182528 14563 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:48.182842 14404 consensus_queue.cc:1059] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:48.161789 14230 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.184755 14185 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:48.187359  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1488) took 1262 ms. Trace:
I20260709 13:05:48.187991  9859 rpcz_store.cc:276] 0709 13:05:46.925233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.930592 (+  5359us) service_pool.cc:224] Handling call
0709 13:05:46.930892 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:46.931098 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:47.703203 (+772105us) log.cc:844] Serialized 14 byte log entry
0709 13:05:48.186353 (+483150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:48.186486 (+   133us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:48.186653 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.186813 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.187120 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":468657,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":1149,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":229,"thread_start_us":1254222,"threads_started":2,"wal-append.queue_time_us":300814}
I20260709 13:05:48.199399  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:47.965983 14451 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:48.173763  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:48.174374 14376 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:48.183198 14563 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:48.157075  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1243) took 1206 ms. Trace:
I20260709 13:05:48.169817 14309 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.203218  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:48.208187  9270 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:48.211351  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:48.213124 14447 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:48.214179 14548 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:48.215288  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.215718  9854 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.000s	sys 0.011s
W20260709 13:05:48.217746  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.164326 14546 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:47.955008 14267 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:47.965466 14348 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:48.182864  9826 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:05:48.240134  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 114330us
I20260709 13:05:48.221140  9861 rpcz_store.cc:276] 0709 13:05:46.950339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.951153 (+   814us) service_pool.cc:224] Handling call
0709 13:05:46.955446 (+  4293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:46.964682 (+  9236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:46.964714 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:46.965142 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:05:47.732297 (+767155us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:47.732357 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:47.732450 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:48.156303 (+423853us) raft_consensus.cc:1685] finished
0709 13:05:48.156357 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.156805 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2840,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":14966,"thread_start_us":727902,"threads_started":1,"wal-append.queue_time_us":977012}
I20260709 13:05:48.243060  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:48.243695  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:48.244108  9861 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:48.244412  9267 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:48.245380  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:48.246008 14534 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:48.246902  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 47915us
I20260709 13:05:48.248711 14525 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:05:48.249191  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:48.251855  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:48.252619  9854 raft_consensus.cc:2376] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16. Re-sending same reply.
I20260709 13:05:48.252060 14548 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.254545 14532 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:05:48.254757  9853 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:48.255024 14532 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:48.255420 14532 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:48.224464 14563 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:48.251353  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:48.256999  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:48.246061  9861 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:48.262882 14426 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:48.263484 14426 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:48.263588 14572 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:48.263975 14426 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:48.265202 14554 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:48.265738 14554 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:48.266165 14554 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:48.227005 14447 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:48.269322  9162 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:48.271086  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:48.262710 14403 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:48.249339 14525 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:48.256536 14115 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:48.232856 14576 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.291412  9267 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
W20260709 13:05:48.289382  9087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.009s	sys 0.025s
I20260709 13:05:48.293406  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:48.294296  9087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.010s	sys 0.026s
I20260709 13:05:48.269362 14447 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.301885 14447 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:48.303016 14302 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:48.297719  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49114.8us
I20260709 13:05:48.290695 14115 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:48.290832 14525 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:05:48.309113 14448 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:48.339396 14448 raft_consensus.cc:697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:48.340128  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.341060 14448 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.294034 14548 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:48.344246  9197 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:48.345492 14552 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:48.345717  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.346053 14552 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:48.346549 14552 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.347296  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:48.348075  9192 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:48.350085 14577 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:05:48.311822 14451 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:48.313736 14414 raft_consensus.cc:3055] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:05:48.350873  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 187965us
I20260709 13:05:48.351058 14414 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:48.352499 14414 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.354264 14414 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:48.334357  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:48.337287 14576 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:48.338562  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 46809.5us
I20260709 13:05:48.349918  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.309584  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:48.356643  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 174021us
I20260709 13:05:48.360054  9162 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:48.373726 14561 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.380198 14403 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.386641 14230 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.388377  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:48.388533  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:48.389386  9191 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:48.390750  9826 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:48.393504 13804 consensus_meta.cc:287] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.759s	user 0.014s	sys 0.004s
I20260709 13:05:48.394604  9861 raft_consensus.cc:2410] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:05:48.396569  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:48.397379  9856 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:48.394111 13804 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:48.366845  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48221.1us
I20260709 13:05:48.323024 14115 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:48.412179  9271 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 18.
I20260709 13:05:48.383483 14403 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.422097 14568 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:48.423233  9190 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:05:48.422919 14568 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.426491 14568 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:48.428287  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 68378.9us
I20260709 13:05:48.431021  9195 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:48.432777  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:48.434676 14451 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:48.435190 14451 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:48.435611 14451 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:48.436236 14563 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.438827 14563 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.448812  9235 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:48.449245 14387 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.449877 14399 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:48.399675 14376 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:48.461733 14399 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:48.462566 14399 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.465129 14399 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:48.466672 14532 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.478797  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:48.481206  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48591.8us
W20260709 13:05:48.487028  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 346742us
I20260709 13:05:48.490041  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:48.490770  9270 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:05:48.458760 14568 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:48.459976 14548 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:48.417363 14529 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:48.504848  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.510861  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.495972 14550 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:48.517035  9856 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:48.497514 14230 raft_consensus.cc:2764] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:05:48.527890 14532 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.520995 14469 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:48.522192  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 3000) took 1158 ms. Trace:
I20260709 13:05:48.512526 14552 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.520999  9862 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:48.535310  9856 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:05:48.519928  9192 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
W20260709 13:05:48.558565  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 62181us
I20260709 13:05:48.549789  9264 rpcz_store.cc:276] 0709 13:05:47.364103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.364917 (+   814us) service_pool.cc:224] Handling call
0709 13:05:47.365245 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:47.767925 (+402680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:47.767992 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:47.775405 (+  7413us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.226388 (+450983us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:05:48.226431 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.226522 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:48.521830 (+295308us) raft_consensus.cc:1685] finished
0709 13:05:48.521852 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:48.522010 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46979,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":502439,"threads_started":2,"wal-append.queue_time_us":433229}
W20260709 13:05:48.574267  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.548139 14550 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:48.582971 14374 consensus_meta.cc:287] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.007s
I20260709 13:05:48.583608 14374 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.586177 14374 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:48.586988 14581 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:05:48.561765 14568 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:48.589598  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.590073  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.590519  9089 connection.cc:256] RPC call timeout handler was delayed by -1.4247386439999445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 13:05:48.590950  9089 connection.cc:256] RPC call timeout handler was delayed by -1.1979386440000326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
W20260709 13:05:48.591348  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0631101079999326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 13:05:48.594002  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 203439us
I20260709 13:05:48.595099  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:48.598871 14399 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:48.600028 14572 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.600997 14572 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.605091  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:48.605651 14552 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:48.605893  9861 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.606741  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:48.607817  9161 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:48.608834 14496 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:48.609359 14496 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:48.610512 13478 raft_consensus.cc:2804] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:48.610407 14496 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.611111 13478 raft_consensus.cc:697] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:48.612555 13478 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.616009  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:48.616647  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.617141  9235 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:48.617895  9122 raft_consensus.cc:2393] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:48.618382 14532 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:48.618867 14561 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:48.619038 14532 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:48.619355 14561 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:48.619493 14532 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:48.619817 14561 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.619689  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:48.621004  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.584039  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:05:48.622637 14200 consensus_meta.cc:287] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 1.085s	user 0.005s	sys 0.013s
I20260709 13:05:48.623967  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:48.625466  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 46622.4us
I20260709 13:05:48.626519  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:48.618309  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:48.628226  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.628835  9235 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:48.630110  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:48.630977 14434 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:48.631392 14434 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:48.631826 14434 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.631928 14563 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:48.632669  9117 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:48.638841  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.639832  9117 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 17: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:48.640942 14451 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.641445  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.640956  9235 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:48.603402 14572 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:48.642330  9117 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:48.642639 14230 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:48.643239 14230 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:48.644384 13804 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.644272 14230 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:48.645720  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.644690  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.648116  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.559228 14469 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:48.649502  9861 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:48.648937  9122 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:48.649976 14469 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.650940  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.651821  9114 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:48.652043  9269 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:48.653021  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:48.654829 14534 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:48.655426  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:48.656339  9114 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:48.656680  9860 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:48.658136  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.658221 14426 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:48.658859  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.658152  9114 raft_consensus.cc:1240] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 13:05:48.587766 14296 consensus_meta.cc:287] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.837s	user 0.010s	sys 0.006s
I20260709 13:05:48.660007  9116 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:48.660140 14296 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:48.661815  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.662755 14296 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:48.664287 14532 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:48.664906  9120 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:48.622514  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.670043  9264 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:48.643428 14451 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.639889 14434 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:48.676573  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 46660.4us
I20260709 13:05:48.677073  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1548) took 1097 ms. Trace:
I20260709 13:05:48.677471  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:48.677745  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:48.678670  9114 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:48.676862  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.680258  9123 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:48.677430  9860 rpcz_store.cc:276] 0709 13:05:47.579769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:47.579897 (+   128us) service_pool.cc:224] Handling call
0709 13:05:48.676860 (+1096963us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540070e4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:48.677008 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2456136576}
I20260709 13:05:48.680855  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:48.682185  9120 raft_consensus.cc:2368] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15.
I20260709 13:05:48.682547 14447 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:48.674937 14387 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:48.582216 14550 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.684862  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:48.684845 14532 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:48.683437 14434 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:48.688552  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.691423  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.696844  9190 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:48.639154 14115 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:48.563268 14554 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.670919 14568 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:48.623204 14200 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.708065  9192 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:48.702147  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:48.730296  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.731051  9115 raft_consensus.cc:2393] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 16: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:05:48.729946  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49708us
I20260709 13:05:48.737711  9263 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:48.737867  9235 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:48.737869  9116 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:48.730265 14532 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:05:48.743683  9235 leader_election.cc:400] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15.
I20260709 13:05:48.744115  9235 leader_election.cc:403] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:05:48.744757 14529 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:48.745198 14529 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:48.745576 14529 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:48.748561  9119 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:48.748883 14451 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:48.749635  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.749872  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.752185  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.752702  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:48.753504  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:48.754308  9123 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:48.754864 14434 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:48.755164 14554 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.755358 14434 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:48.755484  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:48.756194  9825 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:48.756345  9118 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.757112 14302 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:48.761638  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.762972  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.764213  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.765154  9197 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:48.766093  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.768208  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.768141  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:48.769796  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 172487us
I20260709 13:05:48.775758 14561 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:48.739516  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 113166us
I20260709 13:05:48.755642 14426 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:48.783143  9826 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:48.787782  9118 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:48.791127  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:48.796308  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:48.800329  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.804713  9164 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:48.783735 14525 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:48.807241  9265 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
I20260709 13:05:48.808367 14525 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:48.810372  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.810639  9198 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:48.810981  9196 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:48.800220 14568 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:48.812186 14568 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:48.811942  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.812673 14568 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:48.813118  9194 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:48.813064  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.814543  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:48.815254  9855 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 15: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:48.817370  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:48.818099  9118 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:48.812613  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.819643  9195 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:48.812121  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.824958  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:48.851454  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48283.6us
I20260709 13:05:48.756400 14434 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:48.860425  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.861820 14200 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:48.862782 14267 consensus_queue.cc:1059] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:48.859252  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:48.868108  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:48.881883  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:48.902316  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:48.930497  9114 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:48.915011  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.911944 14302 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:48.874864 14451 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15.
I20260709 13:05:48.826790 14426 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:05:48.882763  9162 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:48.876184 14387 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:48.935714  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.936492  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:48.943315  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 60639.1us
W20260709 13:05:48.945508  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 160976us
I20260709 13:05:48.881994 14448 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:05:49.018194 14426 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:49.018733 14426 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:49.019732  9265 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:49.019693 14426 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.021003 14426 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:49.024614  9015 catalog_manager.cc:5697] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.026597  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:49.026937 14447 raft_consensus.cc:3055] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:05:49.028146  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:49.028262  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.028301  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.029160  9195 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:49.029500  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:49.026979  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:48.997786 13683 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:49.031772 13683 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:49.032673 13683 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.033949 13683 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:49.034605  9826 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.545s (SENT)
I20260709 13:05:49.027314  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:49.013787 14594 log.cc:927] Time spent T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:05:49.036430  9018 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36634 (request call id 341) took 34986 ms (client timeout 15000 ms). Trace:
I20260709 13:05:49.014153 14528 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:48.952452  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:48.979444 14461 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:05:49.039584 14461 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.039968 14461 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:49.040728  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:49.036829  9018 rpcz_store.cc:269] 0709 13:05:14.056089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.061202 (+  5113us) service_pool.cc:224] Handling call
0709 13:05:49.036185 (+34974983us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:44.586345 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:44.644789 (+ 58444us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:44.644821 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:05:44.644840 (+    19us) tablet.cc:662] Decoding operations
0709 13:05:44.645724 (+   884us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:44.645782 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:44.645808 (+    26us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:05:44.646449 (+   641us) tablet.cc:701] Row locks acquired
0709 13:05:44.646463 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:05:44.646601 (+   138us) write_op.cc:270] Start()
0709 13:05:44.646741 (+   140us) write_op.cc:276] Timestamp: P: 1783602344646555 usec, L: 0
0709 13:05:44.646760 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:05:44.666293 (+ 19533us) log.cc:844] Serialized 3218 byte log entry
0709 13:05:45.640088 (+973795us) op_driver.cc:464] REPLICATION: finished
0709 13:05:45.985532 (+345444us) write_op.cc:301] APPLY: starting
0709 13:05:45.985618 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:46.085780 (+100162us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:46.096668 (+ 10888us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:46.517079 (+420411us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:46.517407 (+   328us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:05:46.517472 (+    65us) write_op.cc:312] APPLY: finished
0709 13:05:46.517849 (+   377us) log.cc:844] Serialized 81 byte log entry
0709 13:05:46.735850 (+218001us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:46.752457 (+ 16607us) write_op.cc:454] Released schema lock
0709 13:05:46.756074 (+  3617us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:46.764886 (+  8812us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:46.764938 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2182759,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":339019,"apply.run_cpu_time_us":57028,"apply.run_wall_time_us":777447,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37583,"num_ops":8,"prepare.queue_time_us":286974,"prepare.run_cpu_time_us":21769,"prepare.run_wall_time_us":303359,"raft.queue_time_us":504777,"raft.run_cpu_time_us":71160,"raft.run_wall_time_us":563198,"replication_time_us":741430,"thread_start_us":1041331,"threads_started":5,"wal-append.queue_time_us":464705,"wal-append.run_cpu_time_us":200521,"wal-append.run_wall_time_us":2521923}]]}
I20260709 13:05:49.045750 14550 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.046938  9267 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:49.012607 14597 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:05:49.027427 14447 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:49.026640  9015 catalog_manager.cc:5697] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 9, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 9 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:05:49.048846 14447 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.050419 14447 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:05:49.030812 14588 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:05:49.071920 14461 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:49.073241  9194 raft_consensus.cc:2393] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 7: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:49.074478 14461 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:49.075963 14267 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:05:49.078279  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:48.997314 14599 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:05:49.063882  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.050311  9015 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.082332  9015 catalog_manager.cc:5697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 11, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:48.955258 14115 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:49.089246  9015 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 16, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 16 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:05:49.091406  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.086452  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:49.093726  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.093518 13683 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:49.095536  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.095820 14572 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:05:49.095711  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 146820us
I20260709 13:05:49.098779  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.099606 14572 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:49.095515 13683 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:49.093398  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:49.101495  9119 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 6.
I20260709 13:05:49.105728  9235 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:49.106645 14528 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:05:49.107192 14528 raft_consensus.cc:697] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:05:49.113936  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.065526 14434 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:48.947932  9089 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.556s (SENT) [suppressed 19 similar messages]
W20260709 13:05:49.126883  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 472517us
I20260709 13:05:49.128753  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:49.129374  9114 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
I20260709 13:05:49.120645 13683 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:49.130664  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:49.130992  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:48.980557 14302 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:49.131362  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 304316us
I20260709 13:05:49.091262  9015 catalog_manager.cc:5697] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 4 to 5, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 5 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
W20260709 13:05:49.162583  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 63917.2us
I20260709 13:05:49.126268 14461 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:05:49.171355  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 75863.7us
I20260709 13:05:49.180948 14561 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:49.185827  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2288) took 1043 ms. Trace:
I20260709 13:05:49.186347  9272 rpcz_store.cc:276] 0709 13:05:48.142013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:48.450007 (+307994us) service_pool.cc:224] Handling call
0709 13:05:48.450466 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:48.494962 (+ 44496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:48.494998 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:48.495589 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 13:05:48.712114 (+216525us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:48.712173 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:48.712314 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:49.185569 (+473255us) raft_consensus.cc:1685] finished
0709 13:05:49.185595 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:49.185760 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95347,"raft.queue_time_us":45052,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":1189,"thread_start_us":215970,"threads_started":1,"wal-append.queue_time_us":487237}
I20260709 13:05:49.189646  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:49.166109 14230 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:49.129071 14529 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:49.199680 14529 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:49.201335  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:49.201953  9116 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:05:49.198186 13683 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:49.108141 14528 consensus_queue.cc:237] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:49.206670  9825 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.207484  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:49.208761  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.217931 14572 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:49.226310  9116 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:49.204519 14230 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:49.233304  9235 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:49.247567  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 73994us
I20260709 13:05:49.249683  9114 raft_consensus.cc:2393] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:49.251950  9164 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.258603  9119 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:05:49.253367 14591 log.cc:927] Time spent T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:05:49.264441  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 73333.1us
W20260709 13:05:49.265830  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.267524  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:49.255631 14550 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:49.270768  9267 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:49.271842 13683 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:49.272226 13683 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:49.272564 13683 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:49.257840 14534 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:49.260038 14376 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:49.249177 14252 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.135s
I20260709 13:05:49.279788  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:49.275283 14525 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:49.281096 13478 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:49.285809  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.286633  9114 raft_consensus.cc:2393] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:49.234794  9015 catalog_manager.cc:5697] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:05:49.295152  9271 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:49.296636  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:49.281078 14534 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:49.293934 14461 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:49.281283 14376 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.309832  9118 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:05:49.300379 14230 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:05:49.316471 14603 log.cc:927] Time spent T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:05:49.303426 14550 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:49.326681 13683 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.307245 14302 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:49.332826 13683 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:49.335680  9118 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:05:49.340917  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:49.341569  9114 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 6.
W20260709 13:05:49.344811  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 241888us
I20260709 13:05:49.348666  9115 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:49.351977  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:49.363093  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 81015.2us
I20260709 13:05:49.324967 14252 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:05:49.343816 14302 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:49.366709 14595 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.197s	user 0.000s	sys 0.003s
W20260709 13:05:49.367419 14592 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.197s	user 0.001s	sys 0.001s
I20260709 13:05:49.368055 14550 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:49.380335  9120 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:49.382012 14550 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:49.382992  9859 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:49.384083 14534 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:49.386233  9122 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:49.387495  9122 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:49.388087  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:49.388365  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:49.388885  9117 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:49.389850  9122 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:49.391235  9121 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:49.391443 14568 consensus_meta.cc:287] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.578s	user 0.006s	sys 0.004s
I20260709 13:05:49.364332 14374 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:49.388249 14376 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:49.401889  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:49.416303  9120 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:49.421664  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:49.425527  9264 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:49.430871  9265 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:49.432374 14461 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:49.434676 14426 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:49.436359 14252 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:49.437875 14426 raft_consensus.cc:3055] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:49.421712  9192 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:49.440734  9192 raft_consensus.cc:606] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:49.441183  9192 raft_consensus.cc:614] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:49.438642 14115 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:49.442379 14115 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:49.456856  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.469033  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:49.470935  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.470959  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.477725  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.478834  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.480885  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.482270  9195 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:05:49.410071 14374 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:49.486367  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.487303  9192 raft_consensus.cc:2368] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7.
I20260709 13:05:49.444332 14426 raft_consensus.cc:740] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:49.499171  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.402861 14568 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.436786 14252 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:49.500173  9197 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:49.503897  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.507174  9196 raft_consensus.cc:2410] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:05:49.507438  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:49.508049  9191 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.510809  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.512120  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:49.512928  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:49.513468  9114 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:49.443446 14115 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.412422 14376 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
W20260709 13:05:49.516996  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 152466us
I20260709 13:05:49.518512  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.522476  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.525998  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.542294  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.547969  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.547968  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:49.360369  9015 catalog_manager.cc:5697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.503250 14607 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:05:49.558270  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.565565  9198 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:49.566231  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:49.566826  9196 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
W20260709 13:05:49.572170  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.583946  9189 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:49.585752  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 99703us
I20260709 13:05:49.586819  9161 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:49.588665  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 155270us
I20260709 13:05:49.589058 14615 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:49.593923 14448 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:49.593329  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 83239us
W20260709 13:05:49.592998 14605 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 13:05:49.593250 14606 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:05:49.607321  9114 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:05:49.616066  9191 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:49.616752 13478 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:49.565300 14611 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
W20260709 13:05:49.624861  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 106987us
I20260709 13:05:49.626806  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:49.628055  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.630960 14622 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:49.632249  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:49.626365  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:49.572970 14426 consensus_queue.cc:260] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.573225 14252 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:49.616017 14572 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:49.600687  9015 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 6 to -1, term changed from 11 to 15, leader changed from <none> to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) evicted, now has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 15 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:05:49.645500  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:49.646051 14376 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:49.646274  9861 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:05:49.646756 14376 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:49.648927  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:49.649533  9859 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:49.585379 14374 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.641611 14622 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:49.644407 14568 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:49.657739  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:49.626708 14447 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:05:49.662117  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.665961  9861 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:49.664252  9859 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:49.660250 14252 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:05:49.675874  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.686303  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:49.686376 14620 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:05:49.664310 14622 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:49.668721 14387 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.721s	user 0.003s	sys 0.009s
W20260709 13:05:49.692354 14610 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:05:49.693287  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:49.694896 14448 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:05:49.684136 14528 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:49.706400 14532 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:49.706666  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:49.707369  9861 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:49.708184  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.708768  9862 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:05:49.709761 14481 consensus_queue.cc:1059] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:05:49.710459  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.711680  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.712142  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.712402  9856 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:49.712769  9855 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:49.694942 14302 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:49.716169  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.716847  9860 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.717819  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:49.718439  9858 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:49.719411  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.720813  9859 raft_consensus.cc:2368] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15.
I20260709 13:05:49.707026 14532 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.727267  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.721174 14608 consensus_queue.cc:1059] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:05:49.733489  9860 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:49.729272  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.736656  9267 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:49.728309  9264 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:49.740321  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.742013  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.742163  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:49.739069  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.739782  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.743459  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.705202 14387 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:49.745684  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.744182  9267 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:49.743814  9859 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:49.743477  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:49.747972  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 69380.7us
I20260709 13:05:49.749344  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.749403  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.750007  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.750204  9858 raft_consensus.cc:2376] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10. Re-sending same reply.
I20260709 13:05:49.752436  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:49.752792  9266 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:05:49.753073  9856 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:49.749924  9857 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:05:49.746297  9265 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:49.756639  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:49.760361  9193 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:49.759383  9264 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:49.764346  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.765357  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.764994  9270 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:49.767895  9858 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:49.767791  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.777055  9269 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:49.777221  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.778168  9268 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:49.779206  9860 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 16 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:49.700090 14376 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.780249  9856 raft_consensus.cc:1240] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:49.781672  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:49.783174  9272 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 8: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:49.758358 14616 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:05:49.784463  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:49.793550  9862 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:49.759392 14448 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:49.794801  9857 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:49.796221  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 130616us
I20260709 13:05:49.796348  9855 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:49.750211 14461 consensus_queue.cc:1059] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:05:49.750258 14252 consensus_queue.cc:1059] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:05:49.731769 14528 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.794732 14447 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:49.795398 14616 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:49.814906  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.827776  9860 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:49.827482  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.772771 14387 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:05:49.830621  9856 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.831667  9853 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:49.830209  9272 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:49.832461  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:49.800721 14628 raft_consensus.cc:3055] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:49.840960  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:49.841797  9118 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:49.845815  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.848980  9268 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:49.851506 14532 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:49.842590 14447 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:05:49.832199 14616 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:49.781900  9015 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 13 to 15, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 15 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:49.867228  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.868256  9114 raft_consensus.cc:1240] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 6. Current term is 7. Ops: []
I20260709 13:05:49.869455 14447 raft_consensus.cc:697] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:49.869791 14469 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:49.870280 14469 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:49.870550 14447 consensus_queue.cc:237] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:49.871217 14469 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.872110 14115 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.877255  9853 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:05:49.862432  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63597.8us
I20260709 13:05:49.888396 13683 raft_consensus.cc:3055] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:49.888947 13683 raft_consensus.cc:740] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:49.844275 14628 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:49.895798  9089 connection.cc:256] RPC call timeout handler was delayed by -1.0147772259999783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
I20260709 13:05:49.896656 14469 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:49.897346  9860 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:49.900612 14528 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:49.901984  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:49.902602  9268 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.902693  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.904083  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:49.907891  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:49.908038  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:49.908689  9853 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:49.800766 14622 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.913002  9268 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:49.917074  9266 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:49.928349  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:49.928660  9856 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:49.893711 14601 consensus_queue.cc:1059] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:05:49.932247 14572 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:49.939589  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 43635us
I20260709 13:05:49.943819  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:49.973032  9015 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 14, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 14 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:49.978057  9860 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:49.981103 13683 consensus_queue.cc:260] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:49.984062  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.985234  9825 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:49.995890  9267 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:49.998869  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.000072 14628 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.000663  9856 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:50.001554 14628 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:50.002548  9190 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:50.008708  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:50.010046  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:50.010389  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:50.010160 14572 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.010761  9268 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:50.009297  9264 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:50.014644 13683 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:50.017946  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:50.019016  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.021359  9859 raft_consensus.cc:1240] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:50.022869  9853 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:50.035194  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:50.036057  9268 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:50.048168  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:49.995668 14607 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:49.996872 14612 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:50.009286 14616 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.010058 14302 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:50.062057  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.063585  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.064214  9164 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:50.064889  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.051433 14634 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:50.083379  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.084224  9189 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
W20260709 13:05:50.079804  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 67250.9us
I20260709 13:05:50.080735  9194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:50.081122  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.081274  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.088011  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.089273  9193 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:05:50.056849 14615 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:50.085435  9194 raft_consensus.cc:606] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:50.096470  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.097435  9825 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.097630  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.103045  9272 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:50.103291  9123 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:05:50.106487 14568 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:50.106925 14568 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:50.107277 14568 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:50.108142  9269 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:50.053367 14636 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 13:05:50.115065  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.117056  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:50.082991 14634 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:50.137355  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.138370  9189 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
W20260709 13:05:50.124433  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.139678  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.150045  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.068921 14302 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.152830  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:50.159364  9860 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 16 LEARNER]: Advancing to term 17
I20260709 13:05:49.975639  9015 catalog_manager.cc:5697] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:50.160792  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:50.181564  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 116177us
I20260709 13:05:50.099682 14633 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:05:50.147789 14630 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:50.148102 14634 raft_consensus.cc:3060] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:50.194558  9122 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:05:50.185771  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:50.198088  9117 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:05:50.198182 14481 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:50.198681 14481 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:50.199712 14481 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:50.173744 14615 raft_consensus.cc:2764] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:05:50.201282  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.202672  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.203245  9191 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:50.118424 14621 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:50.208976  9264 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:50.156857 14572 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:50.210660  9267 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:50.205123  9114 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:50.207165  9087 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:50.204362  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.203358  9198 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:50.210728  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:50.188320  9263 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:50.213299  9191 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:05:50.215853 14496 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:50.233970  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:50.243053  9196 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:50.259850  9860 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 17 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:50.244364 14631 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:50.245277 14572 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.282126  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.282888  9196 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
W20260709 13:05:50.294284  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 184398us
I20260709 13:05:50.295943  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:50.325165 14652 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 13:05:50.326598 14609 log.cc:927] Time spent T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:05:50.327423 14606 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:05:50.332805  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.334955 14611 raft_consensus.cc:2764] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:05:50.247387 14376 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 13:05:50.318111 14374 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:50.336127 14613 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:50.363487 14613 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:50.364713 14613 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.366230 14613 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:50.337380  9015 catalog_manager.cc:5697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 11, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:50.368237 14550 raft_consensus.cc:3055] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:05:50.368690 14550 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:50.368979  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.368880  9015 catalog_manager.cc:5697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 10, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:50.369645 14550 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:50.370613  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.370985 14550 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:05:50.372244  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:50.340312  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 49051.2us
I20260709 13:05:50.375942  9089 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.376083 14376 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:05:50.378310 14376 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:50.233508 14252 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 13:05:50.340809  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.380820  9192 raft_consensus.cc:2368] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:50.382031  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.385134  9089 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.344429 14627 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:50.337358  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.226379 14621 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.385927  9116 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:05:50.386051 14627 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:50.387914  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.353485 14640 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:50.389614  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.391017  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.353693 14630 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:50.340842 14302 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:50.274089 14572 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.342033 14634 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.362989 14374 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:50.393105 14374 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:50.394100 14572 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:50.220481 14629 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:05:50.240854 14447 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.140s
I20260709 13:05:50.372713  9015 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:50.385013 14568 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.388080 14621 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:50.349707  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.392503 14630 consensus_queue.cc:1243] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:05:50.373327 14448 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:50.344761 14616 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:50.395097  9196 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:50.395498  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 195518us
I20260709 13:05:50.401221  9189 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
W20260709 13:05:50.398757  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 183328us
I20260709 13:05:50.425004  9192 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
W20260709 13:05:50.411991  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 193543us
I20260709 13:05:50.394630 14302 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:50.393106 14387 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
W20260709 13:05:50.432482  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 47425.8us
I20260709 13:05:50.432644  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.433446  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.405323 14448 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:50.443356 14448 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.444867 14639 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:50.399799 14649 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:05:50.394260 14627 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:50.435607 14661 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:05:50.449862  9015 catalog_manager.cc:5697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 15, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 15 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:50.460893  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.462049 14376 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:50.464725  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:50.465621  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 128335us
I20260709 13:05:50.465795 14376 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:50.466434  9826 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:50.464581 14630 consensus_queue.cc:1175] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1efabcb1befc400aadf101e8276fc1aa" responder_term: 13 status { last_received { term: 13 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 13 index: 7 } }
I20260709 13:05:50.467511  9089 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.468222 14302 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:50.468358 14639 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:50.469497 14634 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:50.472091 14629 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.472659 14629 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.475340  9197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49752620cbbf430991915a2400f71286"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:50.475400 14629 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:50.475891 14568 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:50.476064  9197 raft_consensus.cc:606] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:50.476589 14629 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:50.478111  9197 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:50.480536 14115 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:50.482014  9198 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:50.482177 14115 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:50.482945 14115 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.484139 14115 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:50.467561  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.486299  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.481228  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.482959  9194 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
W20260709 13:05:50.480840  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49437.4us
I20260709 13:05:50.493229  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.493957  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:50.500231  9859 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
W20260709 13:05:50.501389  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 61477.3us
W20260709 13:05:50.504262  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.507175  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:50.507951  9858 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 17 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
W20260709 13:05:50.509470  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 275577us
I20260709 13:05:50.509557  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:50.510327  9115 raft_consensus.cc:2393] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:50.512774 14572 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:50.513211 14572 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:50.513567 14572 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:50.512279  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.514815 14639 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:50.515137  9087 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:50.516063  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.470278  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.518179  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:50.518786  9114 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:05:50.519148  9196 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:50.519945  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:50.531814  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:50.462054 14613 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:50.532645  9193 raft_consensus.cc:606] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:50.533702  9193 raft_consensus.cc:614] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:50.534389  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.492471 14374 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:50.537025  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.507670 14568 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:50.541667  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.542313  9196 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:50.510887 14621 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:50.533075 14376 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:50.554368  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 84146.5us
I20260709 13:05:50.555190  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:50.548915 14631 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:50.533247 14665 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:50.546062 14528 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.561820 14528 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.565744  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:50.566655  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.559360  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.542479 14611 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:50.531967 14302 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:05:50.573483 14534 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:50.573689  9015 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 12, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 12 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:50.574919 14653 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:50.575692 14601 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:50.545511 14621 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:50.579646  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:50.581620  9117 raft_consensus.cc:2393] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 19: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:50.580488 14653 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.586527 14653 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:50.587271 14621 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:50.588017  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 51072.3us
I20260709 13:05:50.588788  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.589931  9114 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:50.590071  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.570582 14607 raft_consensus.cc:2764] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:05:50.591056 14658 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:50.591238  9826 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.578925 14611 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:50.591881 14611 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:50.591974 14601 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:50.592506 14601 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:50.594154 14374 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:50.594954  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:50.595525  9193 raft_consensus.cc:606] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:50.549875 14376 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:50.581113 14534 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:50.596092 14376 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:50.596385 14534 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:50.599932 14528 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:50.600265  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45137.2us
I20260709 13:05:50.591276 14607 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:50.601166  9089 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.601325  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.600672  9015 catalog_manager.cc:5697] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 6 to 7, term changed from 6 to 8, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 8 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:50.602523  9089 leader_election.cc:400] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12.
I20260709 13:05:50.603227 14631 raft_consensus.cc:2764] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:05:50.600692 14302 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:50.604059 14631 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:50.604212 14302 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:50.604764 14302 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:50.605743  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.605985 14627 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:50.608654 14481 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:50.595868  9193 raft_consensus.cc:614] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:50.612478 14448 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:50.612960 14572 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.613680  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:05:50.615263 14572 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:50.617080  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:50.621446 14611 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.624891 14426 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:50.626009  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.627218  9826 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:50.627931 14622 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:50.628602 14622 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:50.629067 14622 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:50.629891 14622 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.624820 14658 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:50.580020  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:50.604085  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.649139  9861 raft_consensus.cc:2468] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
W20260709 13:05:50.651437  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 71483us
I20260709 13:05:50.652572  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.583930  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:50.656195  9189 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:50.667716 13478 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:50.666702  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 174559us
I20260709 13:05:50.669875  9164 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.678085  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.685603 13478 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.686013  9189 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:50.682245  9828 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:50.712467  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 138161us
I20260709 13:05:50.673987 14528 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:50.673837 14612 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:50.715116  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 111074us
I20260709 13:05:50.683261 14374 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:50.736467  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:50.718355  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:50.725127 13478 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:50.725008 14612 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.603408  9015 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 6 to 9, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 9 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:50.700855 14653 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:50.700445 14629 raft_consensus.cc:2804] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:50.729617 14658 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:50.729665 14611 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:50.731564 14376 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.734335 14621 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.718277 14528 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:50.807673  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 89394.1us
I20260709 13:05:50.774425 14374 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:50.775910 14658 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:50.862771  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.864182 14496 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.864596  9859 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:50.865021 14496 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.790237 14653 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:50.869908  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:50.803460 14607 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:50.789938 14629 raft_consensus.cc:697] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:50.804754 14528 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:50.875402  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 197424us
W20260709 13:05:50.876293  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 194114us
I20260709 13:05:50.870626  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.877480  9854 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:50.878058  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.890416  9193 raft_consensus.cc:2393] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:50.883363  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.883602  9862 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:50.889358  9189 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:50.888670  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.868498 14658 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:50.883312 14665 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.855365 14632 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:05:50.947645  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:50.895529  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:50.880082 14302 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.928823 14496 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:50.994091  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.995239 14376 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:50.884799 14601 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:50.995671 14632 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:05:50.997211  9089 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:50.998991 14601 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:50.999048  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:50.886562 14684 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 13:05:51.004207  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:50.896103 14653 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:05:51.031410  9859 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:50.992084  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 96674.6us
W20260709 13:05:51.008203  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 60702.4us
I20260709 13:05:51.009534  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:51.037094  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:51.044505  9238 leader_election.cc:400] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15.
I20260709 13:05:51.010128 14679 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:50.958297 14665 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.009690 14601 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:51.016068 14607 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:51.045753  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:51.051256  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:51.036471 14632 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:51.047709  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:51.054435 14679 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.000962  9015 catalog_manager.cc:5697] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 11, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 11 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:51.059274 14679 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:51.059940 14682 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:51.061467 14682 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:51.062031 14665 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.062117 14682 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:51.063139 14682 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:51.063576 14682 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:51.064179  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.066824  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.032157 14683 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:05:51.068272  9191 raft_consensus.cc:2410] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:05:51.069276  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:51.069947  9123 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:51.059226  9015 catalog_manager.cc:5697] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:51.084105  9857 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:05:51.085500  9857 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:51.084990  9015 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 10, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 10 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:51.086589  9015 catalog_manager.cc:5697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:05:51.076514 14676 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:05:51.077136 14667 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:05:51.077510  9854 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:51.078794  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:51.079695 14672 log.cc:927] Time spent T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:05:51.107856 14607 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:51.108809 14607 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.109916  9195 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:51.110543 14662 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.111034 14662 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.055711 14601 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.113466 14612 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:51.119894  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:51.120679  9190 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:51.121913  9190 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:05:51.090637 14674 log.cc:927] Time spent T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.219s	user 0.000s	sys 0.002s
W20260709 13:05:51.053998 14675 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:05:51.091308 14682 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.083496  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:51.127789  9859 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:51.128549 14682 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.073758  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:51.130146 14447 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:51.130616 14621 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:51.130719 14447 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.133203 14447 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.092939 14302 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:51.134826  9828 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:51.097448 14628 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:51.135887 14302 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:51.046841 14681 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:51.136971 14681 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:51.137981 14681 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.139297 14681 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:51.105201 14115 raft_consensus.cc:993] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:05:51.140269 14115 raft_consensus.cc:1081] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:05:51.114379 14601 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.106668  9197 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:05:51.145273  9015 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:51.146306  9123 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:51.115482  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2425) took 1037 ms. Trace:
I20260709 13:05:51.147876  9191 rpcz_store.cc:276] 0709 13:05:50.078388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:50.390742 (+312354us) service_pool.cc:224] Handling call
0709 13:05:51.115265 (+724523us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b54001225d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:51.115404 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1555360128}
I20260709 13:05:51.148234  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:51.118310  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:51.150038  9189 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
W20260709 13:05:51.150658 14686 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:05:51.046811 14629 consensus_queue.cc:237] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.152172  9272 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:51.152751  9089 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:51.153421  9263 raft_consensus.cc:1240] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:51.135923 14628 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:51.142074  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:51.154608  9859 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:51.158453 14662 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.161962  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:51.181188 14376 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:51.181861 14376 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.182301 14623 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:51.183957 14623 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:51.184378 14623 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.184643 14632 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:51.185757 14632 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.188464 14628 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.189749 14628 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:51.189924 14376 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.191931 14631 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:51.054149 14616 consensus_queue.cc:1059] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 13:05:51.164928  9853 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:05:51.196475  9853 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:51.196856  9853 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:51.198710  9089 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:51.165436  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:51.199647 14618 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.200053  9860 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:51.200192 14618 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.202569 14618 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.165547  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:51.203119 14618 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:51.181272  9265 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:05:51.205164  9265 pending_rounds.cc:85] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 4
I20260709 13:05:51.205456  9265 pending_rounds.cc:107] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:05:51.205809  9265 raft_consensus.cc:2889] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:51.211938 14376 raft_consensus.cc:3055] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:51.212414 14376 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:51.213387 14376 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.214614 14376 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:51.207360  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:51.218323  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:51.219151 14618 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:51.219997 14618 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:51.150063 14658 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:51.117707 14534 consensus_meta.cc:287] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.002s
I20260709 13:05:51.221220 14534 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.150316  9270 raft_consensus.cc:1240] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 11. Current term is 12. Ops: []
W20260709 13:05:51.224030  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48490.4us
I20260709 13:05:51.224308 14534 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.245302  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:51.146996  9015 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 5 to 8, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 12 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:51.184751 14623 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:51.165963 14528 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.268285 14658 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:51.268428  9853 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.271010  9853 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:51.271816  9123 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 14. Current term is 15. Ops: []
W20260709 13:05:51.273146  9195 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 1047) took 1799 ms (client timeout 1816 ms). Trace:
I20260709 13:05:51.274487  9116 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:05:51.349848 14629 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:51.276301  9266 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:51.351173 14633 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:51.351594 14633 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:51.351629 14655 consensus_queue.cc:1059] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:51.353052 14630 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:51.353565 14630 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:51.353142  9195 rpcz_store.cc:269] 0709 13:05:49.473409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:49.542097 (+ 68688us) service_pool.cc:224] Handling call
0709 13:05:51.272899 (+1730802us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400398854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:51.273039 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3571386624}
I20260709 13:05:51.354776 14630 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.277287  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.277352 14528 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.279254  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.278208  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:51.283484  9235 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.031s	sys 0.028s
W20260709 13:05:51.357030  9235 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.031s	sys 0.028s
I20260709 13:05:51.357167  9122 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
W20260709 13:05:51.357390  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 643487us
I20260709 13:05:51.357441  9270 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
W20260709 13:05:51.281041  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 62798.6us
I20260709 13:05:51.287577  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:51.359084  9191 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:51.364845  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:51.365628  9855 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:51.366134  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:51.367273  9191 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:05:51.367802  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:51.369841  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.284752 14631 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:51.371842  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:51.296226  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 248689us
I20260709 13:05:51.301148 14689 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:51.292603  9015 catalog_manager.cc:5697] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 6 to 11, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:51.330603  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:51.321280 14615 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.352680 14633 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.373942  9197 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:51.374423  9197 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:51.374598 14633 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:51.375268  9197 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.376758  9197 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:51.372988  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:51.373800 14689 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:51.379987  9858 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:51.380403 14653 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:05:51.381008 14653 raft_consensus.cc:697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:51.380898 14689 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.382134 14653 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.372113  9120 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:51.385735 14615 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.389135 14615 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:51.390377  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:51.046622  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:51.397212  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 350665us
I20260709 13:05:51.396554 14528 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:51.398331 14629 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:51.398936 14629 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.399437 14629 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:51.400604 14528 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:51.401291  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:51.410647  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.409225  9828 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:51.414480  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63144.4us
I20260709 13:05:51.416678  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:51.417397  9191 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:51.416816  9197 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:51.411571  9116 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:51.427004  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.428104  9119 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:51.430379  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.431944  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:51.436404  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:51.472414  9114 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:05:51.463533  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:51.476768  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 57381.8us
I20260709 13:05:51.462616  9121 raft_consensus.cc:3055] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:51.468897  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:51.469002 14690 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:51.485286  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:51.486546 14658 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:51.486497 14690 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:51.468674  9162 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:51.424984 14673 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:51.494598  9162 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:51.478212  9121 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:51.487190  9828 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:51.455435 14302 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:51.462309 14682 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:51.505170  9089 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:51.506214 14374 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:51.506359  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:51.518282  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.489880 14658 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.497314  9121 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.455873 14601 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:05:51.531643  9121 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:51.458535 14649 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:05:51.468279 14550 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:05:51.504532 14681 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:51.487797 14632 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 13:05:51.565819  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.496739 14664 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:05:51.530690 14658 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:51.529961 14621 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.499419 14616 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:05:51.534706 14448 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:51.540067 14302 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:51.520073 14625 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:05:51.523999 14665 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:05:51.503647 14673 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.528379 14682 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:51.606242  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 137431us
I20260709 13:05:51.575265 14639 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:05:51.553512 14662 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:05:51.576269 14681 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 16 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:51.620003  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:51.596122 14621 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.540596 14528 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.626796 14550 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:51.629102 14682 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.630056 14630 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:51.632005 14629 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.635092 14629 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:51.640079  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 133790us
I20260709 13:05:51.641892  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.641978  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:51.615679 14618 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:05:51.647011  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 210563us
I20260709 13:05:51.645182 14715 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:51.618435 14639 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.679333  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.683950  9123 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:51.640717 14448 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:51.690268  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:51.649825 14374 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:51.603176 14623 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.649425 14664 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.704535 14664 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.707031 14374 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.695201 14496 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:05:51.712718 14621 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:51.699810  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:51.714956  9114 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:51.717283  9860 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:05:51.687366 14715 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.688720 14611 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:05:51.687206 14550 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:51.652679 14618 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:51.721505  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.691299 14639 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:05:51.725315  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 230763us
I20260709 13:05:51.693360 14448 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:51.716567  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:51.726198  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 49469.9us
I20260709 13:05:51.734488  9238 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:51.700690 14447 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:51.731568  9193 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:51.732239  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:51.687942 14572 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:05:51.701157  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.740491  9192 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
W20260709 13:05:51.728786 14376 consensus_meta.cc:287] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 13:05:51.743854 14528 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:51.744462 14528 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:51.697093 14629 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:05:51.743136  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:51.729821 14627 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:51.729715 14711 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:51.776181  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:51.790954  9121 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:51.800081  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:51.800737  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 58462.6us
I20260709 13:05:51.742059 14572 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:51.728060 14715 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:51.800877  9116 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
W20260709 13:05:51.813040  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 69977.3us
I20260709 13:05:51.749450  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:51.803608 14675 log.cc:927] Time spent T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:05:51.824357 14623 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:51.792686 14627 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:51.802703 14682 consensus_queue.cc:1059] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:05:51.750134 14601 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:51.803501 14572 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:51.741835 14665 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.083s
W20260709 13:05:51.845363  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 169418us
I20260709 13:05:51.741951 14550 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.847296  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:51.851194  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:51.851945  9858 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 17 LEARNER]: Advancing to term 18
I20260709 13:05:51.823614 14621 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:51.734706 14302 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.855206  9853 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:51.858913  9121 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
I20260709 13:05:51.858253  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:51.860252  9193 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:51.738931 14681 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.839479 14673 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.844551 14627 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:51.799211 14711 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:51.896557  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 104927us
I20260709 13:05:51.897084  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.897168  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.898067  9116 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:51.845064 14601 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:51.833509 14612 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:05:51.902145  9121 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:51.856650 14621 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:51.906183  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:51.907058  9853 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:51.839236 14653 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:05:51.907495 14664 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.925262 14550 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:51.925987 14550 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:05:51.903990 14601 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:51.910801  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:51.912933 14711 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:51.919533 14621 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:51.929834 14633 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:51.900977 14689 raft_consensus.cc:3055] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:05:51.936583 14722 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:05:51.937336 14710 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.235s	user 0.000s	sys 0.000s
I20260709 13:05:51.939476  9193 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:05:51.944996  9858 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 18 LEARNER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
I20260709 13:05:51.892970 14649 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.946101  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 100843us
I20260709 13:05:51.842128 14618 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.981673  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:05:51.947923  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 91176.5us
I20260709 13:05:51.951620  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:51.951598  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:51.933126 14708 log.cc:927] Time spent T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
W20260709 13:05:51.959574  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.964217 14655 raft_consensus.cc:993] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:05:51.963814 14633 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.947636 14550 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:51.966135 14713 raft_consensus.cc:3055] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:52.019635  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:52.024652  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:52.027308  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:52.021337  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:51.840888 14448 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:51.985121  9122 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:51.983830  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:51.978084 14496 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:51.965041 14725 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:51.989651  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:51.989861  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:51.991508  9191 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:52.007091  9858 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:05:52.038645  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43907.9us
I20260709 13:05:52.038614  9195 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:52.016577  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:51.997836 14550 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:51.957702 14689 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:05:52.046818  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 63082.9us
I20260709 13:05:51.980527 14628 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:51.945837 14607 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:05:52.057853 14615 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:52.057879  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.059250  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.060106  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.060310  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.060909  9198 raft_consensus.cc:2376] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12. Re-sending same reply.
I20260709 13:05:52.061178  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.061762  9190 raft_consensus.cc:2376] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12. Re-sending same reply.
I20260709 13:05:52.056917  9195 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:52.066347 14621 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:52.037266  9087 leader_election.cc:336] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.518s (SENT)
I20260709 13:05:52.078411  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:51.987104 14632 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:51.986106 14534 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.041996  9853 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:52.020408 14618 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:52.050282 14721 raft_consensus.cc:1081] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:05:52.050458 14633 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.055915 14447 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.056880 14628 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.057219  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.029673 14711 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:52.034099  9196 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:05:52.029982 14673 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:52.022027 14713 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:52.086575  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.034629 14717 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.092837  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 43710us
I20260709 13:05:52.040378 14374 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:05:52.126634  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:52.091941 14632 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:52.114152  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 76928.4us
I20260709 13:05:52.071285 14658 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.108834  9193 raft_consensus.cc:2376] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14. Re-sending same reply.
I20260709 13:05:51.985356 14649 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.040771 14496 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:52.103442  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 44754.1us
I20260709 13:05:52.107059 14740 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:52.122048  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:52.046087 14550 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:52.137943  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:52.138597  9853 raft_consensus.cc:2410] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:05:52.142146  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:52.108588 14730 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:52.108157 14534 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.118273 14673 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:05:52.150790  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.117815 14611 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.108732 14732 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.150069  9087 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.553s (SENT)
I20260709 13:05:52.159157  9861 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:52.130779 14720 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:05:52.159821  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:52.111914 14448 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:52.131963 14627 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.120306 14628 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.163815  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:52.164207  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:52.166452  9854 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 18 LEARNER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:05:52.167007  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:52.212205  9853 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:52.168875  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:52.113056 14447 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.110931 14719 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.182992  9087 leader_election.cc:336] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.592s (SENT)
I20260709 13:05:52.220625  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:52.188972  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:52.221709  9858 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:52.164855 14615 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:52.190707  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 53673.5us
I20260709 13:05:52.117004 14625 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:05:52.135666 14665 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:05:52.214483  9857 raft_consensus.cc:2393] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:52.214267  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:52.221519  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 77011.9us
I20260709 13:05:52.213279  9862 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:52.115963 14717 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:52.235409 14695 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.061s	user 0.000s	sys 0.005s
I20260709 13:05:52.241973  9192 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:52.245625  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 62665us
I20260709 13:05:52.253692  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.254153  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.255645  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:52.256454  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.257213  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:52.257557  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.258615 14745 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:52.258845  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.260066  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.260771  9197 raft_consensus.cc:2376] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10. Re-sending same reply.
I20260709 13:05:52.261264 14632 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.240172 14682 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.262372 14632 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:52.262631 14682 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:52.263842 14738 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:05:52.264639 14711 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.265180 14682 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.265143 14711 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.267241 14711 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.260100  9115 raft_consensus.cc:1275] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:52.260185 14737 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:05:52.223214 14628 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.271425  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:52.271885  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:52.275821  9198 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:05:52.287917  9853 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:52.180162 14611 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.294988  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:52.295641  9196 raft_consensus.cc:2376] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12. Re-sending same reply.
I20260709 13:05:52.259159 14745 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.157320 14689 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.243034 14528 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:52.308187 14673 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:52.311936  9195 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:52.312395  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:52.301645 14748 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 13:05:52.296599 14639 consensus_meta.cc:287] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.555s	user 0.000s	sys 0.009s
I20260709 13:05:52.189390 14732 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.290858 14302 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.321441  9272 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49752620cbbf430991915a2400f71286"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:05:52.323841  9272 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:52.332603  9272 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:52.333922  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.334978 14534 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.301903 14628 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.337255 14628 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.305344 14572 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:52.339440 14715 consensus_meta.cc:287] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.000s
I20260709 13:05:52.334124 14627 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.232852 14719 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.348496  9190 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:52.342128 14715 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:52.352434  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 70622.5us
I20260709 13:05:52.357066  9272 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.375695  9272 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:52.347770 14658 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:52.377131  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.382349  9265 raft_consensus.cc:2410] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:05:52.384770  9189 raft_consensus.cc:2393] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:52.358814 14621 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:52.358925 14723 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.390039  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.418751  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:52.421622  9267 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:52.390861  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.391935 14751 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.392416 14753 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:52.332036 14302 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.425352  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.395952 14741 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:05:52.396039  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:52.396158  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.325796 14639 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.397300 14447 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:52.397918  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:52.384189 14655 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.433465  9117 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 17. Current term is 18. Ops: []
I20260709 13:05:52.400614 14627 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.401099  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.416626  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:52.426249  9115 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:52.427189  9087 leader_election.cc:336] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.613s (SENT)
I20260709 13:05:52.435429  9123 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:52.426258  9267 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:52.399303 14745 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.386592 14633 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.387986 14649 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.399892 14717 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:52.436949  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.438221  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:52.440201  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.461745  9272 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:52.465230  9193 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:52.443231  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.439599  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:52.451077  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:52.452364  9859 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:05:52.455942  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.457334  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.460822  9265 raft_consensus.cc:2376] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12. Re-sending same reply.
I20260709 13:05:52.359423 14615 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.475306  9271 raft_consensus.cc:2376] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12. Re-sending same reply.
I20260709 13:05:52.471982  9270 raft_consensus.cc:2376] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12. Re-sending same reply.
I20260709 13:05:52.493829  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:52.496639  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 57119.9us
I20260709 13:05:52.497750  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.498589  9271 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:52.499609  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.497750  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:52.501971  9862 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:52.503067  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:52.499538  9115 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:52.503962  9198 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:52.506253  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:52.506824  9194 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:52.507197  9194 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:52.509220  9197 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:52.511284  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.511927  9265 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:52.512012  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 56275.1us
I20260709 13:05:52.513211  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:52.521616  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.522337  9270 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:52.507930  9194 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.527051  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.527855  9272 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:52.528885  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.527902  9267 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:52.533354  9194 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:52.497004 14621 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:52.425555 14673 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.445709 14550 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.444304 14627 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.426574 14723 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.462141 14633 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.512481 14745 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:52.552147  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.551863  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.551249 14611 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:52.525072 14720 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:52.624109  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.625268 14719 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:52.631485  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.432498 14751 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.438141 14655 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.655762  9265 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
W20260709 13:05:52.641637  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:52.659415  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.642385  9194 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:52.434743 14753 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.648468  9263 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:52.644335 14658 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:52.649307  9272 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:05:52.649259  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 125151us
W20260709 13:05:52.652875  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 100437us
I20260709 13:05:52.544406 14621 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:52.551541 14715 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.665977  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.669746  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:52.672542  9270 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 8.
I20260709 13:05:52.525503 14534 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.115s
I20260709 13:05:52.698072  9271 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:52.675279  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.707947  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.687394  9272 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:52.713711  9269 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:52.713995  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.690901  9196 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:52.644829 14745 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.676717  9267 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:05:52.670856  9089 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:52.714471  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.730944  9272 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:52.716478  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:52.715770  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:52.730810  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 60072.7us
I20260709 13:05:52.733161  9264 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:52.731050  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:52.735004  9267 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
W20260709 13:05:52.674572 14601 consensus_meta.cc:287] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.747s	user 0.012s	sys 0.000s
I20260709 13:05:52.740725  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.743016  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:52.742425  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:52.744885 14755 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.224s
I20260709 13:05:52.748919 14754 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:52.704952 14682 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:52.749456  9272 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:52.695046 14732 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:52.693581 14723 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:52.662119 14631 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:05:52.743621  9269 raft_consensus.cc:2376] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15. Re-sending same reply.
I20260709 13:05:52.752676 14733 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:52.697028 14751 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.736070 14720 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:52.685248 14550 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.749233 14752 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:52.680253 14633 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:52.759418  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:52.856731 14628 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.108s
I20260709 13:05:52.953676  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:52.954349  9862 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:52.956907 14755 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:52.957548 14755 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.959868 14655 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.857267 14723 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.960758 14627 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:52.961706 14627 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:52.962280 14627 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.860147 14733 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.963171 14733 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:52.964079  9269 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:52.964557 14627 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.762372  9271 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:52.965258  9269 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
W20260709 13:05:52.966025 14618 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.856s	user 0.011s	sys 0.000s
I20260709 13:05:52.966261  9271 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:05:52.744359 14534 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 94504d29c6744f38bb06943479df9466 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:52.752770 14745 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:52.766657 14632 consensus_meta.cc:287] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.002s
I20260709 13:05:52.917948 14720 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:52.969439 14720 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:52.837905 14639 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.746935 14601 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:52.937614  9272 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:05:52.717900 14658 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.962608 14753 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:52.968741 14765 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:52.977272 14765 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.979341 14755 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:52.983382  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:52.936115 14754 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:05:52.960803 14723 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.990430  9269 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:52.993537 14601 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:52.979501 14658 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.006096 14720 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.987442 14723 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:52.979373 14765 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.014580 14733 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:52.683811 14621 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.096994 14621 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:53.076989  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.083712 14682 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:53.084414 14720 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.084654  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:53.087234  9862 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:05:53.076296 14690 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:05:53.100472  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 351590us
I20260709 13:05:53.115958  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.117141  9860 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:53.118494  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:53.117877 14690 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:53.120415 14690 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:53.118779  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:53.128360  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.129550 14682 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.133380 14740 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.134873 14733 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.136793  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:53.140944  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.127476 14720 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:05:53.144675  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.147292 14775 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.149385  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.142632 14733 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:53.144989 14740 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:53.153309 14775 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.154331 14720 raft_consensus.cc:697] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:53.161392  9860 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:05:53.165620  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.167899  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.169613  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.171146 14752 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.171670 14752 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.174290  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:53.174911  9115 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.179343 13683 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:53.179891 13683 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:53.180709 13683 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.181697 13683 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:53.187332 14572 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:53.190088 14689 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:53.191319  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38874 (request call id 2440) took 1201 ms. Trace:
I20260709 13:05:53.199229  9115 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:53.202700 14628 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:53.203336 14628 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.205260  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.205468 14628 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.206045  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 3077) took 1062 ms. Trace:
I20260709 13:05:53.200616  9189 rpcz_store.cc:276] 0709 13:05:51.989527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.989642 (+   115us) service_pool.cc:224] Handling call
0709 13:05:53.191033 (+1201391us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b5400398854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:53.191233 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":794168576}
I20260709 13:05:53.206379  9266 rpcz_store.cc:276] 0709 13:05:52.143912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.144783 (+   871us) service_pool.cc:224] Handling call
0709 13:05:53.205148 (+1060365us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfeee 0000000000375939 00007f765b6331e0 00007f765b6fc0c4 00007f765b6fc131 00007f765b6aa0a0 00007f765b6a9984 00007f7654c246d4 00007f7654c24662
0709 13:05:53.205956 (+   808us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2620288}
I20260709 13:05:53.208241 14690 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:53.211301 14621 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.200124 14711 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.216171 14711 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.218295 14711 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:53.220325 14764 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.594s	user 0.013s	sys 0.000s
I20260709 13:05:53.223147  9265 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:53.224318 14720 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:53.229204  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.232903  9267 raft_consensus.cc:1240] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 10. Current term is 11. Ops: []
W20260709 13:05:53.233755  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:53.233929  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.242242  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:53.242856  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.242905  9263 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:05:53.244709  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:53.245329  9268 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:53.248142 14665 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.252823 14665 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.255268 14665 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.260480  9235 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:53.264595  9268 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:05:53.266040  9263 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
W20260709 13:05:53.269600  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 133697us
I20260709 13:05:53.271916  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.272930  9263 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:53.274746  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:53.251390 14616 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.281585 14572 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.292306 14616 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.292780 14572 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.294485 14616 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.294874 14572 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.294987 14773 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.281411 14621 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.295536 14773 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.295809 14621 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.298043 14773 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.285789 14726 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.298875 14726 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.281842 14689 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.300508 14689 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.302313 14726 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.284977 14528 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:53.303329 14528 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:53.304840 14528 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.306756  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.307962 14611 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:53.308262 14622 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.308678 14611 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.309654 14615 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.310245 14615 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.307351  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:53.312474  9854 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:53.313062 14615 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.313613  9856 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:53.313457  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.313831  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.275859 14612 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.267096 14665 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.314620  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.314922 14612 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.315068 14665 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.315861  9853 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:53.317389  9121 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2803) took 1326 ms. Trace:
I20260709 13:05:53.317539 14612 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:53.308831 14622 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.285481 14711 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.318338 14711 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.298091 14621 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.319693 14622 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.320667  9854 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:53.321007 14711 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.311388 14611 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.323150  9162 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:53.317866 14665 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.323871  9270 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:05:53.324676  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:05:53.324998  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 1.33681e+06us
I20260709 13:05:53.307832 14658 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.325759 14658 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.328933 14752 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.329118 14630 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:53.329514 14630 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.329862 14630 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:53.325182 13683 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:53.333657  9164 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:53.333925 13683 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.334281 13683 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.332520  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 1072) took 1470 ms. Trace:
I20260709 13:05:53.314657  9858 raft_consensus.cc:2410] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:05:53.307878  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.332517  9162 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.340340  9862 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:53.341859 13683 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.344023 13683 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.335975  9192 rpcz_store.cc:276] 0709 13:05:51.862136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.862335 (+   199us) service_pool.cc:224] Handling call
0709 13:05:51.862713 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.242615 (+379902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.332254 (+1089639us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54004173d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:53.332423 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864954112}
I20260709 13:05:53.346262  9853 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:05:53.347793  9270 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:05:53.347808  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:53.349155  9862 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:53.357756  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.317793  9121 rpcz_store.cc:276] 0709 13:05:51.990792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.990899 (+   107us) service_pool.cc:224] Handling call
0709 13:05:51.991145 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.120536 (+129391us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:05:52.120828 (+   292us) log.cc:844] Serialized 14 byte log entry
0709 13:05:52.295446 (+174618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:52.295498 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:52.295545 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.317162 (+1021617us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400253d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:53.317207 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:53.317309 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11779,"prepare.queue_time_us":2485,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":15460,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":8721,"spinlock_wait_cycles":297036544,"thread_start_us":174223,"threads_started":2,"wal-append.queue_time_us":98782,"wal-append.run_cpu_time_us":2183,"wal-append.run_wall_time_us":1108111}
I20260709 13:05:53.359126  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.358433  9853 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:53.302878 14689 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:53.359022  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.384603  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.385047  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.385286  9121 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:53.387410 13683 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.387945 13683 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.359148 14426 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:05:53.354801 14658 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.402120  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.402998  9116 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:53.354810 14662 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.405820  9123 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:53.361783 14773 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.410250  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.383555 14714 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.391789  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:53.357829 14302 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.418025  9862 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:53.417672  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.419072  9115 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:53.419173  9164 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:53.395865 14682 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:53.396274  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:53.397743  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.399217 14633 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.389655  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:53.421710  9263 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.409053  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.411115  9119 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:53.423844  9858 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:05:53.391708  9826 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.357319 14736 consensus_queue.cc:1048] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:05:53.415800 13683 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.417632  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:53.427063  9123 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:53.409682 14681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.196s	user 0.003s	sys 0.061s
I20260709 13:05:53.430729  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.357620 14752 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:53.421449  9120 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:53.428753  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.433590 13683 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.437309 13683 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.375269 14374 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:53.420776 14682 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.411312 14426 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:53.445098 14752 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.424926 14662 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:53.453611  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.430192 14689 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:53.425513 14629 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:53.434741 14681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.004s	sys 0.061s
W20260709 13:05:53.460328  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 219768us
I20260709 13:05:53.454988  9263 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:53.463085 13683 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.441154 14611 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:53.453074 14655 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:53.418560 14773 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.471236  9162 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:53.435825 14730 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:05:53.472750  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.483863  9123 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:53.462400 14662 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.477963  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:53.478758  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.420991 14714 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.495466  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:53.496019  9161 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.479900  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.445358 14682 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:53.485844 13683 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:53.426782 14633 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.492377  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.493265  9116 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:53.494627  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:53.449986 14302 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:05:53.499450  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 80363us
I20260709 13:05:53.451252 14374 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.496635 13683 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.503149  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.468129 14689 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:53.505064  9161 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:53.463100 14572 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:05:53.481439 14611 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:53.486799 14627 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:53.521399  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.523142 14426 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.524358 14611 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.526659  9087 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:53.512936  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.524279  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.529438  9116 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:05:53.501204 14681 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:53.529568  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:53.516835  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.489773 14723 raft_consensus.cc:2764] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:05:53.518617 13683 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.565660 13683 raft_consensus.cc:2804] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:53.566061  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.505120  9271 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:53.492959 14662 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:53.482455 14655 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.537113  9115 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:53.539731 14633 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.543329 14720 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:05:53.544245  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.573336  9235 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:53.574358  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.548552 14616 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:53.549621 14664 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:53.550657 14628 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:05:53.511556 14733 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:53.555183 14787 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.529568 14726 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.482079 14629 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.566273 13683 raft_consensus.cc:697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:53.589651 13683 consensus_queue.cc:237] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.512141  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.504546 14731 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:53.593076 14731 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.577185  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.595178  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.595193 14731 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:53.588197  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.596426  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:53.596746 14730 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:53.597566  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.599009  9238 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.600076 14730 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.599853 14689 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.604192  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.605156 14730 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:05:53.608500  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 108241us
I20260709 13:05:53.608850  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.609920  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:53.574195 14746 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:05:53.599879  9264 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:53.615403  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.586938 14633 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:53.624269  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 119276us
W20260709 13:05:53.653209  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.616283  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.661118  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.617887 14733 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:53.633366 14746 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:53.591010 14787 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.677734  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.613876 14730 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:05:53.662320 14740 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.617254 14775 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:05:53.685345  9264 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:05:53.618932 14711 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:05:53.659078 14633 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.672523 14627 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:53.683158 14752 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.699867  9114 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:53.698266  9264 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:05:53.700464  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 85180.5us
W20260709 13:05:53.701109  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 86589.2us
I20260709 13:05:53.701632  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.714874  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:53.715790  9191 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:53.736528  9238 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.737620 14740 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.738216  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.743690  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:53.744184  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:53.699033 14714 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.744657  9192 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:53.744809  9189 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:53.694324 14791 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:05:53.750077  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 97583.1us
I20260709 13:05:53.751240  9161 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:53.702805 14773 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:53.739727  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:53.754606  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.755221  9193 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:53.756212  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.756405  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.757230  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:53.757534 14778 log.cc:927] Time spent T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: Append to log took a long time: real 0.514s	user 0.002s	sys 0.002s
I20260709 13:05:53.757907  9197 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:53.758520 14627 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.705812 14374 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:53.765267  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.765513  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.769635 14496 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.770529 14682 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:53.773154 14790 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:05:53.775293 14655 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.738247 14740 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.780170  9198 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:05:53.745415 14754 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:53.749185 14653 raft_consensus.cc:3055] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:53.739964 14689 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:05:53.751573 14795 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:53.793957  9263 raft_consensus.cc:2410] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:05:53.757916 14717 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:53.794871  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.793270 14673 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:53.807549  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:53.793049 14653 raft_consensus.cc:740] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:53.809147  9264 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:53.798831 14717 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.812849  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.750844 14713 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.766358 14746 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.823844 14629 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.825419 14633 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:53.827944  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2328) took 1380 ms. Trace:
W20260709 13:05:53.834599 14796 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:05:53.826661 14713 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.818027 14673 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:53.839445 14673 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.814919 14717 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:53.828364  9269 rpcz_store.cc:276] 0709 13:05:52.447988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.966877 (+518889us) service_pool.cc:224] Handling call
0709 13:05:52.967135 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:52.990971 (+ 23836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:53.827613 (+836642us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b54004452d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:53.827869 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53133056}
I20260709 13:05:53.784555  9191 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:53.800767 14795 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:53.844955  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 93796.3us
I20260709 13:05:53.915174  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.916616  9191 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 16: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:53.848596  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:53.937165  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 88680.8us
I20260709 13:05:53.939859  9189 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:05:53.940171  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:53.946436  9190 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:53.946614  9198 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:05:53.945569  9197 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:05:53.942811  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:53.950227  9193 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:53.940876 14627 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:53.953994  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:53.956931 14799 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 13:05:53.958220  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:53.951269  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.936746 14633 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:53.964023  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.968838  9198 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:53.964493  9193 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:53.971848  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.979530  9120 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:53.980609  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:53.980935 14625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.353s	user 0.006s	sys 0.013s
W20260709 13:05:53.981491 14625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.353s	user 0.006s	sys 0.013s
I20260709 13:05:53.975606  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.976378  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.953214 14627 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:53.981411  9116 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:53.982252  9118 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:53.982167  9114 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:53.976500  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.973556  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:53.984498  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:53.985131  9117 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:53.987075 14726 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:53.821262 14653 consensus_queue.cc:260] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:53.993732  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.994117  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.994537  9859 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:53.994793  9855 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:53.995596  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.993639  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.997103  9862 raft_consensus.cc:2393] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:53.999193  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:53.999228  9828 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:53.999994  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.001736  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:54.002720  9197 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:54.004537  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.005326  9854 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:53.996213  9857 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:54.005991  9828 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:54.008127  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.009727  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.006856  9164 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:54.013589  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:54.014923  9828 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:54.016561  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:54.017277  9858 raft_consensus.cc:3060] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:54.017498  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:54.018908  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.011858 14568 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:53.983758 14689 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:05:54.020793 14568 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.021229 14568 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:54.024039 14658 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:54.025297  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.025635 14723 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:54.025794  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.026122 14723 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.026535 14723 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:54.026753  9190 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
W20260709 13:05:54.027901 14630 consensus_meta.cc:287] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.698s	user 0.005s	sys 0.016s
I20260709 13:05:54.028347 14630 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:54.030550 14630 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:54.011420 14732 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:05:54.035354 14732 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.036401  9238 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:53.977638 14633 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:53.997691 14627 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:54.045353  9858 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:54.057883  9197 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:05:54.049206  9164 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:54.012854 14787 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:54.075009  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:54.087815  9117 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.037428 14803 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:54.061761  9190 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:05:54.037585 14755 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.047573 14732 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:54.088128  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:54.056075 14607 raft_consensus.cc:2804] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:54.092280 14607 raft_consensus.cc:697] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:54.093261 14607 consensus_queue.cc:237] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.008087 14681 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:05:54.097719 14681 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:54.098100 14681 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:54.098896 14681 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.061713 14731 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:54.100394  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38650 (request call id 1176) took 1708 ms. Trace:
I20260709 13:05:54.100800  9270 rpcz_store.cc:276] 0709 13:05:52.392003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.673418 (+281415us) service_pool.cc:224] Handling call
0709 13:05:54.100232 (+1426814us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54001c5cd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:54.100340 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1339310336}
I20260709 13:05:54.103142  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.103989  9195 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:54.106344  9115 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:54.107820 14809 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.056378 14690 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:54.108371 14809 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.110613 14809 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:54.114225  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 3087) took 1474 ms. Trace:
I20260709 13:05:54.114626 14633 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:54.056555 14682 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:54.114652  9265 rpcz_store.cc:276] 0709 13:05:52.639687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.674918 (+ 35231us) service_pool.cc:224] Handling call
0709 13:05:54.114049 (+1439131us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54001c61d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:54.114164 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1171732352}
W20260709 13:05:54.123090  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.112046 14690 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:54.056924 14658 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:54.126080  9828 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:54.060395 14630 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:54.088228 14629 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:54.132598  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:54.132930  9117 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:54.090356 14803 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.140383  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.145867 14568 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.146539 14723 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.150076 14568 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:54.150801 14568 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:54.157274  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:54.158018  9263 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
W20260709 13:05:54.158286  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.158965  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:54.159533  9263 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.100420 14731 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.126494 14376 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:54.162204 14376 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:05:54.162395  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.132750 14572 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:54.163058 14376 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.163831 14803 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:54.164094 14376 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:54.164448  9858 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:54.140744 14629 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.168227 14629 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.171272  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:54.143586 14690 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:54.125710 14787 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:54.163058 14572 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.173024 14572 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.125706 14755 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.174253  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:54.174402  9267 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2313) took 1780 ms. Trace:
I20260709 13:05:54.161880 14731 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:54.175230  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:54.186326  9266 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:54.186656  9117 raft_consensus.cc:2368] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15.
I20260709 13:05:54.180629  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.181319  9263 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:54.181488  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.182199  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.188265  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.188680  9197 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:54.189220  9195 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:54.150903  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.183714  9267 rpcz_store.cc:276] 0709 13:05:52.394046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:52.740173 (+346127us) service_pool.cc:224] Handling call
0709 13:05:53.232616 (+492443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.233711 (+  1095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.174200 (+940489us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54003a25d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:54.174336 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079696128}
I20260709 13:05:54.192682  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.195127  9195 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:54.194420  9193 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:54.140849 14681 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:54.199290  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.198230  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.206971  9190 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:54.203346  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.228034  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:54.229952  9089 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:54.231488  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.205108  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:54.237358  9195 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:05:54.237792  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.238901  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.239691  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:54.240306  9267 raft_consensus.cc:2376] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14. Re-sending same reply.
I20260709 13:05:54.223959 14737 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:54.242005 14737 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.198900 14630 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:05:54.247650 14630 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.247853  9235 leader_election.cc:400] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15.
I20260709 13:05:54.248241  9235 leader_election.cc:403] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:05:54.248172 14630 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:54.250676  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:54.208320 14662 consensus_meta.cc:287] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.628s	user 0.015s	sys 0.004s
I20260709 13:05:54.253094  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:54.254047  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.255407  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.257050  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:54.257625  9119 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:54.257687  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:54.262272  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:54.262944  9116 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:54.263952  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:54.264320  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:54.266098  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:54.269102  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:54.270362  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:54.270982  9117 raft_consensus.cc:3060] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.236263 14809 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.273981  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:54.274427  9861 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:05:54.269925  9116 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:05:54.280947  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.283270  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.251825 14752 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:54.289070  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:54.289511  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:54.294258  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.294545  9122 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:54.295264  9859 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
W20260709 13:05:54.296116  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.293102  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 162881us
I20260709 13:05:54.296250  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:54.263085 14665 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:54.260679 14787 raft_consensus.cc:2764] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:05:54.291657  9089 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:54.264982 14633 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:54.253209 14740 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15.
I20260709 13:05:54.309156  9267 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:05:54.316488  9114 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:54.354243  9859 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:05:54.361019  9122 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:05:54.361406  9122 pending_rounds.cc:85] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 13
I20260709 13:05:54.361634  9122 pending_rounds.cc:107] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:05:54.362551  9122 raft_consensus.cc:2889] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:54.360548  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:54.320335  9117 raft_consensus.cc:2468] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:54.321486  9119 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:05:54.265189 14662 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.371592  9856 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:54.304867 14665 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:54.402719  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:54.390076  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.322523 14737 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:54.387485  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:54.359062 14752 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:54.424877  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.430174  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 192837us
I20260709 13:05:54.361636 14633 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:54.365767 14795 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:54.322729 14773 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:05:54.434669  9122 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:54.299923 14809 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:05:54.436592  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49180.6us
I20260709 13:05:54.403895 14665 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:54.406528 14810 log.cc:927] Time spent T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:05:54.437606  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:54.400110 14426 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.445178  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:54.445520  9161 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:54.445866  9272 raft_consensus.cc:2376] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17. Re-sending same reply.
W20260709 13:05:54.444391  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 239366us
I20260709 13:05:54.406406 14658 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:54.449025  9268 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:05:54.411667 14723 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:54.450395  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:54.457005  9192 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:54.461001  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:54.466143  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 240173us
I20260709 13:05:54.469349  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:54.484545  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:54.482844  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:54.443369 14765 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:54.495285  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:54.444078 14752 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:54.498451  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:54.501237  9268 raft_consensus.cc:1275] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:54.444816 14754 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:54.448442 14795 raft_consensus.cc:2764] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:05:54.452629 14792 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:54.494411 14731 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:54.520619  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:54.528179  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:54.529417  9087 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:54.453784 14426 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:54.455408 14787 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:05:54.506747 14736 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:54.498796 14765 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:54.538264  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.547508  9826 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:54.516649 14792 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:54.518777 14754 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:54.591814 14800 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:54.522537 14633 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:54.544162  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:54.539440  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 178964us
I20260709 13:05:54.586205  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:54.545166 14814 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:05:54.492704 14758 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:05:54.548053 14787 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:05:54.599035  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 54984.4us
I20260709 13:05:54.537575 14726 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:54.508884 14629 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.511034 14803 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.546286 14736 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:54.548717 14765 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:54.588888 14731 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:54.609717  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:54.610774  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:54.592208 14717 consensus_meta.cc:287] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.645s	user 0.003s	sys 0.007s
I20260709 13:05:54.619889  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:54.545166 14658 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.595372 14792 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:54.645231  9197 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:54.605752 14754 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:54.608196 14766 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:54.607653 14611 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:54.617393 14736 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:54.675568  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.676527  9194 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:54.653301 14733 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.665611  9825 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:05:54.686260  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 78706.5us
I20260709 13:05:54.687480  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:54.605993 14732 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.611881 14720 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
W20260709 13:05:54.672405 14627 consensus_meta.cc:287] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.597s	user 0.005s	sys 0.004s
I20260709 13:05:54.674052 14662 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:54.625653 14717 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:54.701022  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 202619us
I20260709 13:05:54.672570 14832 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:05:54.765465  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 145628us
I20260709 13:05:54.757072  9196 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 11: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:54.771519  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:54.721824  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:54.744918  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 57559.8us
W20260709 13:05:54.805935  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:54.771762  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:54.809162  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:54.736398  9268 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:05:54.702333 14820 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:05:54.817126  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:54.641295 14752 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:54.819082  9191 raft_consensus.cc:1240] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 12. Ops: []
I20260709 13:05:54.790876  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:54.831508  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 60066.5us
I20260709 13:05:54.833617  9826 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:54.697499 14766 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.825995  9269 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:54.821908 14820 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:54.757465 14746 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:54.789860 14773 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:05:54.872097  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 55103us
W20260709 13:05:54.873561  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 115747us
I20260709 13:05:54.874290  9825 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:54.772487 14607 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:54.701730 14627 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:54.823815 14809 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:54.883716 14827 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.241s	user 0.000s	sys 0.007s
I20260709 13:05:54.884696 14611 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:54.868546 14766 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:54.885440 14611 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:54.886005 14832 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:54.883848 14622 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:54.887734 14622 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:54.888767 14622 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:54.889910 14822 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.251s	user 0.000s	sys 0.004s
I20260709 13:05:54.873865 14773 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:54.892453 14773 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:54.895068 14627 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:54.899950 14843 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:54.900702 14843 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:05:54.902721 14653 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:05:54.899477 14812 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.546s	user 0.003s	sys 0.000s
W20260709 13:05:54.872084 14823 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:05:54.947813  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:54.693953 14733 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:54.956058  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:54.872267 14820 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:54.956486  9115 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:05:54.959337  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 125811us
I20260709 13:05:54.959419  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:54.887593 14766 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.012461  9860 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:55.012643  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:55.018769  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 71056.6us
W20260709 13:05:54.955133  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:54.951237 14681 consensus_meta.cc:287] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.002s
W20260709 13:05:54.963861  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 457632us
I20260709 13:05:54.940474 14809 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:54.953393 14720 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:05:55.027261  9115 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:55.023703  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.025099  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:55.025373  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:55.029209  9861 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:55.011235 14625 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:55.032510  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:55.033254  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:54.927100 14746 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.040782  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:55.042690  9271 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.050370  9860 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:55.042834  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:55.018894 14629 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.030172 14766 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:05:55.084914  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.106281  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:55.027630 14723 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:55.027716 14374 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:55.034423 14713 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:55.039448 14625 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:55.039063 14689 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:55.044701 14658 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:55.047912 14843 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:55.130347  9826 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:55.063504 14630 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.801s	user 0.010s	sys 0.001s
I20260709 13:05:55.055867 14376 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:55.130601  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2339) took 1337 ms. Trace:
I20260709 13:05:55.149839  9269 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:05:55.048919 14842 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
W20260709 13:05:55.141898  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.119374 14835 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.146191  9264 rpcz_store.cc:276] 0709 13:05:53.793019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.811216 (+ 18197us) service_pool.cc:224] Handling call
0709 13:05:53.818280 (+  7064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:53.818671 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:53.818704 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:53.948831 (+130127us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.187236 (+238405us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:54.248845 (+ 61609us) log.cc:844] Serialized 14 byte log entry
0709 13:05:54.251066 (+  2221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.251146 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:54.370148 (+119002us) spinlock_profiling.cc:243] Waited 7.67 ms on lock 0x7b18004b66a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f7653571c0a 00007f765ccb1219 00007f765a5b858f 00007f76535f7571 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd
0709 13:05:54.898726 (+528578us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:54.964817 (+ 66091us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b18004b66a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:55.130321 (+165504us) raft_consensus.cc:1685] finished
0709 13:05:55.130350 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.130525 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64236,"prepare.queue_time_us":64316,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":11959,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":23982,"spinlock_wait_cycles":49388928,"thread_start_us":191203,"threads_started":1,"wal-append.queue_time_us":289555}
I20260709 13:05:55.113129 14766 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:55.134388  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 108348us
I20260709 13:05:55.055631 14681 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:05:55.112308 14848 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:55.160627  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2378) took 1222 ms. Trace:
I20260709 13:05:55.060880 14809 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:55.128796 14732 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.175077  9191 rpcz_store.cc:276] 0709 13:05:53.938190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.938587 (+   397us) service_pool.cc:224] Handling call
0709 13:05:53.952943 (+ 14356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.824910 (+871967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.157370 (+332460us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b540010ca54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:55.160538 (+  3168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1991285504}
I20260709 13:05:55.131407 14713 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:55.201004  9116 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
W20260709 13:05:55.175093 14834 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 13:05:55.164887 14766 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.152979 14717 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:55.218307 14853 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 13:05:55.174652 14837 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:05:55.131989 14374 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:55.141362  9271 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:55.155650 14850 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:55.181020 14658 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:55.146270 14689 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.164505 14723 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 13:05:55.221046 14841 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:05:55.150964 14630 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:55.201771 14831 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.341s	user 0.000s	sys 0.001s
W20260709 13:05:55.221652 14828 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:05:55.163866 14376 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:55.271195  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 89701.8us
I20260709 13:05:55.273061  9193 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:05:55.168965 14835 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.181571 14848 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:55.283177 14731 consensus_meta.cc:287] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.657s	user 0.001s	sys 0.012s
I20260709 13:05:55.283689 14731 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:55.285662 14713 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.288765 14733 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:55.290344 14755 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:55.290575  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.293282 14848 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
W20260709 13:05:55.271806 14852 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:05:55.204018 14625 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:55.296758 14746 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:55.297942 14572 consensus_meta.cc:287] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.124s	user 0.007s	sys 0.006s
I20260709 13:05:55.298352 14572 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:55.271353 14840 log.cc:927] Time spent T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
W20260709 13:05:55.299957 14846 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 13:05:55.303841  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.304630  9264 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:05:55.305732  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.306442  9266 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:55.287523 14689 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.295228 14773 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:55.308928 14765 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.682s	user 0.007s	sys 0.004s
W20260709 13:05:55.328523  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 165154us
I20260709 13:05:55.330754  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:55.331696  9122 raft_consensus.cc:2410] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:05:55.330212  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:55.333336  9115 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 18.
I20260709 13:05:55.333807  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:55.334201  9853 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:55.335819  9123 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 1024) took 1069 ms. Trace:
I20260709 13:05:55.338188  9122 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:55.336189  9123 rpcz_store.cc:276] 0709 13:05:54.266001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.266609 (+   608us) service_pool.cc:224] Handling call
0709 13:05:54.266988 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.298602 (+ 31614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.298648 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.324444 (+ 25796us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.538841 (+214397us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:54.538889 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.538963 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:55.335602 (+796639us) raft_consensus.cc:1685] finished
0709 13:05:55.335625 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.335768 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16268,"raft.queue_time_us":16465,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":33033,"thread_start_us":154788,"threads_started":1,"wal-append.queue_time_us":334163}
I20260709 13:05:55.340443  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:55.343887 14860 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:05:55.343990 14630 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.347662  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 1566) took 1143 ms. Trace:
I20260709 13:05:55.349880  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.350489  9266 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:55.348137  9862 rpcz_store.cc:276] 0709 13:05:54.203907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.231138 (+ 27231us) service_pool.cc:224] Handling call
0709 13:05:55.347464 (+1116326us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54006013d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:55.347606 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1817601792}
I20260709 13:05:55.337404  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:55.352500  9238 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 2.132s (SENT) [suppressed 43 similar messages]
I20260709 13:05:55.329691 14765 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.345118 14809 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.338024 14689 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:05:55.363487  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:55.386425 14665 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.933s	user 0.006s	sys 0.003s
I20260709 13:05:55.386890 14665 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.397153  9118 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:55.397720 14665 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.398566 14376 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.363499 14572 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.399569 14374 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.400957 14773 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:55.402649 14765 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:55.404138 14736 consensus_meta.cc:287] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.000s
I20260709 13:05:55.404923 14835 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.405787 14766 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:55.406302 14792 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.755s	user 0.009s	sys 0.001s
I20260709 13:05:55.406769 14792 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:55.408706 14854 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.148s	user 0.004s	sys 0.000s
I20260709 13:05:55.411057 14792 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.397415 14820 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.413012 14820 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.404589 14736 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.416352  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.406088 14611 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:55.419384  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.417950  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:55.366091 14867 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:55.423184 14867 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.423563 14867 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:55.423620  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:55.424302  9269 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.426281 14736 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.419610  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:55.427595  9853 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:05:55.427613  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.428563  9189 raft_consensus.cc:1240] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:55.428799  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:55.429312  9264 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:55.421298  9266 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:55.421640  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:55.422540  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:55.433521  9123 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:55.422360 14376 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:55.433506  9862 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:55.442185  9264 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:55.447067  9089 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:55.444590 14752 consensus_queue.cc:1059] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:05:55.455523 14374 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:55.435233 14766 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.457125 14374 raft_consensus.cc:3055] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:55.457501 14374 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:55.458333 14374 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.460004 14374 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:55.451361  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:55.466785  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:55.468331  9269 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:55.472757  9123 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:55.474454  9122 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:55.475768  9123 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:55.475963  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:55.476666  9115 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:55.477464  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:55.478165  9114 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:05:55.478385  9266 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 20.
I20260709 13:05:55.481200  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.464632 14843 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:55.469909 14803 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.476977 14766 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:55.471278 14765 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:55.493323  9115 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:55.493347  9114 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 20.
I20260709 13:05:55.495151  9266 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:55.487329 14665 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:55.512149 14665 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:55.514389 14665 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.515516  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:55.516920 14829 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.659s	user 0.003s	sys 0.000s
I20260709 13:05:55.521380  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2381) took 1582 ms. Trace:
I20260709 13:05:55.521802  9194 rpcz_store.cc:276] 0709 13:05:53.939285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.940844 (+  1559us) service_pool.cc:224] Handling call
0709 13:05:54.676179 (+735335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.677093 (+   914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.521180 (+844087us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b54000a7b50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:55.521320 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1690123648}
I20260709 13:05:55.526518  9861 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 1573) took 1250 ms. Trace:
I20260709 13:05:55.526932  9861 rpcz_store.cc:276] 0709 13:05:54.275658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.280641 (+  4983us) service_pool.cc:224] Handling call
0709 13:05:55.526348 (+1245707us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b5400511654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:55.526459 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1718948224}
I20260709 13:05:55.529449  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 1095) took 1541 ms. Trace:
I20260709 13:05:55.529848  9198 rpcz_store.cc:276] 0709 13:05:53.988461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:53.994373 (+  5912us) service_pool.cc:224] Handling call
0709 13:05:53.994689 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:53.994906 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:05:54.455284 (+460378us) log.cc:844] Serialized 14 byte log entry
0709 13:05:54.648283 (+192999us) log.cc:844] Serialized 14 byte log entry
0709 13:05:54.905949 (+257666us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b1800258e80. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:55.528927 (+622978us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b1800258e80. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f7653571c0a 00007f765ccb1219 00007f76535f9a40 00007f76535f44e4 00007f76535f4455 00007f76566dc060 00007f76566ee716 00007f76566ff287 00007f76566ff208
0709 13:05:55.529097 (+   170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:55.529150 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:55.529200 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.529277 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.529394 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194,"prepare.queue_time_us":727804,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":73,"raft.queue_time_us":492,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":10511,"spinlock_wait_cycles":119946624,"thread_start_us":651576,"threads_started":2,"wal-append.queue_time_us":316083}
W20260709 13:05:55.533311 14816 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.039s	user 0.003s	sys 0.009s
I20260709 13:05:55.534832  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2341) took 1738 ms. Trace:
W20260709 13:05:55.535073  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 763348us
I20260709 13:05:55.535238  9270 rpcz_store.cc:276] 0709 13:05:53.796393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.118289 (+321896us) service_pool.cc:224] Handling call
0709 13:05:54.118615 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.118969 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.118997 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.128060 (+  9063us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.271337 (+143277us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:05:54.271392 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.271474 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:55.532516 (+1261042us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:55.534531 (+  2015us) raft_consensus.cc:1685] finished
0709 13:05:55.534554 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.534773 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7467,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":2314,"spinlock_wait_cycles":11239168,"thread_start_us":122512,"threads_started":1,"wal-append.queue_time_us":254380}
I20260709 13:05:55.534826 14794 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:55.539366 14794 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:55.539805 14794 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:55.541831 14633 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:55.542342 14633 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:55.545336 14866 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:55.545918 14866 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:55.546541  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.535959 13683 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:55.548267 13683 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:55.548609 13683 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:55.484776 14746 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:55.550316  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 651165us
I20260709 13:05:55.550464 14746 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.558741  9270 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:05:55.493340 14765 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:55.567610 14765 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:55.489655 14766 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.576232 14766 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:55.460819 14867 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:55.574450 14825 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:55.574764 14633 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.580154  9161 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:55.575495 14866 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.568001 14655 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:55.580404 14825 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.581532 14867 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:55.582113 14655 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:55.582433 14825 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.582858 14633 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:55.583600  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:55.584391  9268 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 18. Current term is 19. Ops: [16.11-18.12]
I20260709 13:05:55.586585  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:55.587100  9115 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:55.588672 14655 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.560643 13683 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.591233  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:55.591800  9195 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:05:55.592087  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 96343.1us
I20260709 13:05:55.593617  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.594142  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.596444  9828 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:55.597010  9198 raft_consensus.cc:1240] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:55.599890  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:55.600109  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:55.602576  9860 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:05:55.604710  9860 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:55.605880  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:55.606683  9122 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:55.607106  9861 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:55.606035  9853 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:05:55.609803  9861 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:55.610659  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:55.610970  9860 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:55.611195  9123 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.612767  9858 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:05:55.606014  9164 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:55.620323  9198 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:05:55.621771  9115 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 19.
I20260709 13:05:55.623100  9235 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:55.627825  9191 raft_consensus.cc:1275] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:55.627942  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.624830  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.626854  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.626586  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:55.631492  9115 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.596562 14496 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.632519 14496 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:55.634764 14496 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.637939 13683 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.638144  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:55.620051 14723 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:55.639305  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:55.639830  9862 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 11: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:55.640394  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.630954 14825 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:55.648494 14843 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.648754 14794 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.649470  9861 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:55.598634 14842 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:05:55.660872 14653 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:55.661725 14673 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:55.661813 14653 consensus_queue.cc:1059] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:05:55.625607  9122 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:55.663565  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:55.664055  9123 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
W20260709 13:05:55.666664  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43671.2us
I20260709 13:05:55.670192  9123 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:05:55.672994  9861 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:55.662366 14673 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.671207  9195 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:55.676060  9162 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:55.677727  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 1101) took 1473 ms. Trace:
I20260709 13:05:55.681315  9193 rpcz_store.cc:276] 0709 13:05:54.203949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.205796 (+  1847us) service_pool.cc:224] Handling call
0709 13:05:54.206147 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.273484 (+1067337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.677412 (+403928us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54004178d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:55.677654 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2453132544}
I20260709 13:05:55.681922  9195 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:55.666183 14653 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:55.624547 14836 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:55.589264 14655 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.684340  9115 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:55.689743 13683 consensus_queue.cc:1048] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:55.692226  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:55.671978 14689 raft_consensus.cc:3055] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:05:55.692989  9115 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:55.695940  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:55.673451 14496 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.698191  9266 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:05:55.672134 14835 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:05:55.699725  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:55.700948  9270 raft_consensus.cc:3060] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.699960  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:55.678664 14869 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.699003 14689 raft_consensus.cc:740] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:55.716056  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.716549 14689 consensus_queue.cc:260] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.717257 14496 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:55.717881 14689 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:05:55.697028 14673 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.718967 14611 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:55.674361 14682 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:05:55.720062 14611 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:55.720477 14611 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:05:55.719076 14858 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.499s	user 0.002s	sys 0.000s
I20260709 13:05:55.687595 14653 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:55.654000 14866 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:05:55.721359 14611 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.722486 14611 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:55.722270 14653 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.723318 14653 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:55.723206 14765 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.726190  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.727509  9121 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:55.727967  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.728176 14867 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:55.693549 14836 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.713933 14655 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.689519 14825 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:55.684748 14842 consensus_queue.cc:1059] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:05:55.666785 14848 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:05:55.729940  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:55.730288  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.690593 14766 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.737680  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.763633  9263 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:55.742398  9270 raft_consensus.cc:2468] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:55.742578  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.725373 14765 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.746379 14723 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:05:55.747215 14818 log.cc:927] Time spent T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.236s	user 0.003s	sys 0.000s
I20260709 13:05:55.748246  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 1106) took 1263 ms. Trace:
W20260709 13:05:55.749717  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.754911  9825 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:55.761966  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:55.776700  9855 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:55.706089 14791 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:55.780618  9825 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:55.765605  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:55.766181  9264 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:55.775048  9190 rpcz_store.cc:276] 0709 13:05:54.484710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.487349 (+  2639us) service_pool.cc:224] Handling call
0709 13:05:54.510180 (+ 22831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:54.512443 (+  2263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:54.512485 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:54.512525 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:05:54.512553 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:55.747949 (+1235396us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b54000eef54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:55.748038 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.748182 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4771584}
I20260709 13:05:55.766403 14726 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:55.769300 14878 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:05:55.769608 14658 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:55.769351 14832 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:55.792613  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.716598 14426 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.776957  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.777313  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.749363 14627 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:55.784179  9089 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:55.783079 14792 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.789978  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:55.772035 14835 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:55.772146 14836 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.788715  9162 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:55.816475  9193 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:05:55.815244  9089 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:55.824640  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:55.825526  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:55.826087  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:55.820772  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:55.814297  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:55.815868  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.823064  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.826598  9164 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:55.828114  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.830494  9089 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:55.831452  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.786818 14726 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:55.828037  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.834777  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:55.836007  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:55.837325 13683 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:55.767163 14682 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:05:55.837546  9161 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:55.841522  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 52871.8us
I20260709 13:05:55.842949  9162 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:55.793032 14843 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:55.829177 14627 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.844245 14843 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:55.845649 14627 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.828785 14720 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:55.845521 14843 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.846477 14720 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:55.846726 14665 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:55.846915 14720 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:55.853950  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1195) took 1714 ms. Trace:
I20260709 13:05:55.854452  9265 rpcz_store.cc:276] 0709 13:05:54.139657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.140150 (+   493us) service_pool.cc:224] Handling call
0709 13:05:54.152498 (+ 12348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:54.152841 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:05:54.152868 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:54.153301 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:05:54.270941 (+117640us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:54.271012 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:54.271104 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:55.746204 (+1475100us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:55.839234 (+ 93030us) raft_consensus.cc:1685] finished
0709 13:05:55.839261 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:55.853869 (+ 14608us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1050,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"spinlock_wait_cycles":15957760,"thread_start_us":117351,"threads_started":1,"wal-append.queue_time_us":289020}
I20260709 13:05:55.792819 14791 raft_consensus.cc:697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:55.823352 14842 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.792840 14832 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:55.859308  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.832513 14851 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:05:55.872251  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.833261 14788 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:05:55.829221 14809 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:55.886698 14815 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:55.832597 14726 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:55.858438 14836 raft_consensus.cc:2804] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:55.833487 14752 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:55.887542 14815 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.891024  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:05:55.892015 14880 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:05:55.892903  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:55.815284 14658 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.893448 14766 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.895133 14658 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.895664 14794 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.870173  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 1102) took 1401 ms. Trace:
I20260709 13:05:55.896281  9196 rpcz_store.cc:276] 0709 13:05:54.468802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.469002 (+   200us) service_pool.cc:224] Handling call
0709 13:05:55.870025 (+1401023us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b5400417dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:55.870126 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634200832}
I20260709 13:05:55.839135 14755 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:05:55.897140 14835 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:55.838801 14496 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:55.841789 14607 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:55.898417 14815 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:55.898850 14607 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:55.875891 14855 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:55.719496 14869 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.838687 14426 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:05:55.901453  9195 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:05:55.908334  9855 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:05:55.881963 14856 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:55.840724 14630 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.846486 14795 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:05:55.903759  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 51237.8us
W20260709 13:05:55.905117  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 62259.1us
I20260709 13:05:55.829938 14792 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.787106 14376 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:55.916957  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.920506  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:55.922050  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 84688.7us
I20260709 13:05:55.922904  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:55.924372  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:55.921824  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.941730 14794 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:55.944190 14788 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:05:55.944356 14885 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:05:55.853137 14825 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.945851 14792 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.951002 14746 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:55.959801  9196 raft_consensus.cc:1240] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11. Ops: []
I20260709 13:05:55.953876  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56646 (request call id 1104) took 1470 ms. Trace:
I20260709 13:05:55.954073  9192 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:05:55.910207 14821 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:55.955888  9194 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:55.907930 14851 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 20 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:55.964854  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:55.926162 14836 raft_consensus.cc:697] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:55.961588  9197 rpcz_store.cc:276] 0709 13:05:54.483970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.484173 (+   203us) service_pool.cc:224] Handling call
0709 13:05:55.953677 (+1469504us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54004164d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:55.953818 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358857984}
I20260709 13:05:55.961028 14496 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:55.902429 14842 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:55.974397  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 221592us
I20260709 13:05:55.909932 14803 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:05:55.968549  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.915864 14630 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.953799 14878 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:55.968714 14867 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:05:55.982769  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:56.001889  9270 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:55.984407  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:55.969385  9197 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:56.006629  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:55.968201 14821 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:55.963716 14788 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:55.983394 14795 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:55.964494 14832 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:55.985637  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:55.984457 14791 consensus_queue.cc:237] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:55.988890  9263 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:05:55.993256 14884 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:55.915496 14720 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:56.025157  9235 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:56.016217  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 1020) took 1755 ms. Trace:
W20260709 13:05:56.025390  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 189447us
I20260709 13:05:56.025985  9118 rpcz_store.cc:276] 0709 13:05:54.260556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.260869 (+   313us) service_pool.cc:224] Handling call
0709 13:05:54.275431 (+ 14562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.397591 (+1122160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.015981 (+618390us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001e4e54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:56.016151 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2579204352}
I20260709 13:05:56.026947 14832 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:56.027302 14803 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:56.022732  9270 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:56.011795 14821 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:05:56.034830  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 49333.6us
I20260709 13:05:56.036190  9089 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:56.035955  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:56.045478  9195 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:56.023121 14884 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:56.021965 14795 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:56.041332 14689 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:05:56.056448 14892 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:56.057015 14892 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:05:56.057394 14857 log.cc:927] Time spent T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.838s	user 0.001s	sys 0.003s
I20260709 13:05:56.057264  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:56.059001  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.047176 14866 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:56.053112  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:56.059643  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:56.045559 14891 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:56.075250  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.076400  9856 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:56.077145  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.078013  9195 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:56.079205  9855 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:56.081756  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:56.087189  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.087904  9118 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:56.043540 14836 consensus_queue.cc:237] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:56.090754  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.091173  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.052711 14848 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:05:56.057436 14892 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:56.076017 14891 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:56.098977  9089 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:05:56.080667 14856 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:05:56.122821  9114 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:56.123898  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.098264  9087 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:56.124526  9122 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:56.125476  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:56.125988  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.111874  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:56.126761  9860 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.078896 14884 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:56.113059 14896 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:05:56.133266  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.115563  9189 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 1105) took 1631 ms. Trace:
I20260709 13:05:56.134089  9189 rpcz_store.cc:276] 0709 13:05:54.484358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:54.485555 (+  1197us) service_pool.cc:224] Handling call
0709 13:05:54.514992 (+ 29437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.428934 (+913942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.115321 (+686387us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b54001d5454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:56.115506 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2100599552}
I20260709 13:05:56.040007 14851 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.117445  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:56.119872 14869 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:56.086833 14765 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:56.123214  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.125346 14665 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:05:56.140440  9116 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:56.122794 14720 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:56.144897  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:56.104728  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.094642 14766 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:56.092203 14887 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:05:56.129545  9123 raft_consensus.cc:1240] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:56.129418  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:56.131335  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:56.132529  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.138389  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.104641 14891 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.057850 14788 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.100497  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:56.123915 14856 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:05:56.228451  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 128022us
I20260709 13:05:56.229534  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:56.141557 14665 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:56.233841  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:56.225095  9860 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
W20260709 13:05:56.213372  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 115138us
I20260709 13:05:56.228745 14792 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:56.259642  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:56.219977  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 90661.4us
I20260709 13:05:56.182687 14884 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.241335  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:56.241643 14665 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.244961 14754 raft_consensus.cc:2764] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:05:56.245826 14736 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:05:56.180470 14766 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:56.222741 14376 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:56.218106 14630 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:56.230635 14856 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:56.180785 14773 consensus_queue.cc:1059] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:05:56.265360  9087 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:56.143185 14842 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:56.236567 14690 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:56.268846 13683 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:56.269351  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.185081 14607 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.297571  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:56.305488  9189 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:56.276576 14736 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:56.308049 14895 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
W20260709 13:05:56.311269 14833 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.439s	user 0.000s	sys 0.007s
I20260709 13:05:56.313640 14607 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:56.315047  9267 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:56.316633  9858 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:56.316891 14726 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.318852 14726 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:56.321681  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:56.324178 14899 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:05:56.324553  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:56.325685  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:56.326022  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:56.326642  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:56.326838 14815 raft_consensus.cc:3055] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:56.327504 14815 raft_consensus.cc:740] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:56.326984  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:56.329232  9189 raft_consensus.cc:1240] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:56.306385  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:56.329710  9114 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:56.329152  9190 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:56.276180 14866 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:05:56.337776  9190 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:56.333389  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:56.334955  9123 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:05:56.335666  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.278185 14903 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:56.279297 14766 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.340062  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.303938 14752 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:56.299256 14795 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:56.350251  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:56.279341 14803 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:05:56.329275 14690 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.374519  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:56.302359 14792 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:05:56.377002  9860 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:05:56.377867  9825 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:56.354882 14726 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:05:56.369732 14690 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:05:56.397539  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 115915us
I20260709 13:05:56.393873  9089 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:56.401909  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:56.407959 14903 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:56.408659 14903 raft_consensus.cc:697] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:56.409775 14903 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.388597 14607 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:05:56.412454 14607 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:56.412873 14607 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:56.413743 14607 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.414865 14607 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:56.414631 14855 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:05:56.415726  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.416050 14855 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:56.416543 14855 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:56.417600 14855 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.419097 14835 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.403317  9855 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:56.388720 14875 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:56.419459 14855 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:56.442276  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 88514.7us
I20260709 13:05:56.400682 14726 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:56.371301 14815 consensus_queue.cc:260] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.449723  9087 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:56.446272  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 44431.8us
I20260709 13:05:56.419639 14835 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.459733  9161 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:56.446245 14792 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:56.462904 14792 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:56.463243  9161 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:56.446434  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:56.401181 14901 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:05:56.466846  9161 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:56.457885 14773 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.457506 14752 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:56.478837 14915 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:56.479444 14915 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:56.480405 14915 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.481894 14737 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:56.485415  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.490127  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:56.491247  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:56.494760  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:56.496693  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.507376  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:56.469151 14791 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:56.470683 14792 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:56.512434  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:56.472258 14815 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.513093  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:56.513116  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:56.518200  9196 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:56.515898  9198 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.522862  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:56.508878 14653 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:56.470533 14803 consensus_queue.cc:1059] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:05:56.511914 14791 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.533188 14791 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.533722 14627 consensus_queue.cc:1059] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:56.533964 14821 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:56.536728  9123 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:56.514288 14625 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:56.513583 14905 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:56.545058  9198 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:56.546402  9089 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:56.541169  9193 raft_consensus.cc:2368] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 15. Current term is 16.
I20260709 13:05:56.534823 14627 raft_consensus.cc:3055] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:56.543359  9123 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:56.550386 14821 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:56.493340 14773 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.519608 14733 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:56.511338 14376 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:05:56.530251 14869 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:56.573514 14910 log.cc:927] Time spent T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:05:56.529001 14866 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:05:56.560700 14737 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:05:56.555297 14627 raft_consensus.cc:740] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:56.543764 14689 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:05:56.515240 14726 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.598037 14927 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:56.599291 14773 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:56.599359 14835 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:56.605834 14918 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:56.608579 14737 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:56.609851  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.617200  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.617810  9122 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:56.618777  9825 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:56.619455 14737 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:56.615000 14888 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:56.620599 14888 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:56.621411 14888 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.622578 14888 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:56.692811  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:56.695050  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:56.695863  9123 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:56.701893  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.702744  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:56.703166  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.703871  9861 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:56.704039  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.704865  9855 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 18 LEARNER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:05:56.705121  9854 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:56.702764  9856 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:56.708276  9122 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:56.709896  9122 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:05:56.718142  9122 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:56.568809 14752 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.608924 14627 consensus_queue.cc:260] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.722962  9859 raft_consensus.cc:1240] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:56.727006  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.727839  9856 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
W20260709 13:05:56.705209 14921 log.cc:927] Time spent T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:05:56.642117 14773 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:56.724260  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:56.757603  9164 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:56.763389 14826 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.764073 14826 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.765574 14689 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:56.766108 14689 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:56.766835 14826 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:56.768399 14920 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.133s	user 0.002s	sys 0.000s
W20260709 13:05:56.773419 14892 consensus_meta.cc:287] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.001s
I20260709 13:05:56.773887 14892 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:56.765719 14924 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:05:56.767028 14689 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:56.720832 14825 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:56.610400 14791 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.786492  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:56.789016  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:56.796262  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:56.796557  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.790663  9123 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:56.790730 14426 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:56.791175 13683 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:05:56.792119  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:56.799610  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:56.792357  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.757507 14773 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:56.795545 14884 consensus_meta.cc:287] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.000s
I20260709 13:05:56.785255 14496 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:56.789078 14825 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:56.794427 14627 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:56.794849 14821 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:05:56.755018 14766 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:56.804883 14843 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:56.815133 14773 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.815729 14794 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:05:56.794253 14826 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:05:56.861508  9268 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 3113) took 1822 ms. Trace:
I20260709 13:05:56.822170 14825 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:56.857033  9266 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 3128) took 1140 ms. Trace:
I20260709 13:05:56.886297  9266 rpcz_store.cc:276] 0709 13:05:55.716861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.727114 (+ 10253us) service_pool.cc:224] Handling call
0709 13:05:55.727428 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:55.954519 (+227091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:55.954568 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:55.959191 (+  4623us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.036490 (+ 77299us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:56.036538 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.856411 (+819873us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b540032e454. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:56.856501 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:56.856573 (+    72us) raft_consensus.cc:1685] finished
0709 13:05:56.856594 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:56.856944 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31373,"raft.queue_time_us":27716,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":18141,"spinlock_wait_cycles":482134400,"thread_start_us":41101,"threads_started":1,"wal-append.queue_time_us":113074}
I20260709 13:05:56.874696 14689 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:05:56.875825 14924 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:05:56.883451 14929 log.cc:927] Time spent T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:05:56.806555 14875 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:05:56.869133  9268 rpcz_store.cc:276] 0709 13:05:55.039610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.043467 (+  3857us) service_pool.cc:224] Handling call
0709 13:05:55.414094 (+370627us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:55.584733 (+170639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.083053 (+498320us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54002a3d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:56.861218 (+778165us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54002a3d50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:56.861423 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241445376}
W20260709 13:05:56.828971 14891 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.599s	user 0.011s	sys 0.000s
I20260709 13:05:56.820653 14496 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:56.868634 14890 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:05:56.896543  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.866629 14794 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:05:56.897315  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 97777.7us
I20260709 13:05:56.827562 14892 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:56.836750 14800 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:05:56.916618  9189 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:05:56.917054  9189 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:56.917513  9191 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:05:56.917833  9189 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.919376  9189 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:56.919945 14794 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:56.903239 14736 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.920946 14736 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.901494 14690 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.924409 14884 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:56.932603  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.933425 14607 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:56.934006 14607 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:56.936064  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 131395us
I20260709 13:05:56.938196  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:56.944177  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:56.919584 14825 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.954536 14825 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:56.955869  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.957302 14496 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:56.958377 14791 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:56.955688 14825 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:56.909536 14891 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:56.961937 14891 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:56.962919  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:56.924933 14884 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:56.968704  9189 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:05:56.969154  9189 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 11
I20260709 13:05:56.941867 14800 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:56.969823  9189 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:05:56.936560 14607 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:56.970837  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 126662us
I20260709 13:05:56.971174  9189 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:56.971660  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:56.973779  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:56.976745  9116 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:56.980489  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:56.983613  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:05:56.989100  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:56.986124  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:56.994117  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:57.000288  9116 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:57.010833  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:57.012823 14914 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:05:57.035079  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 90972.4us
I20260709 13:05:57.038447  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:05:57.043005  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.023223 14766 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:57.064296  9856 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:57.068228 14892 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:05:57.072384 14794 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.053586 14820 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.055867 14902 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.681s	user 0.004s	sys 0.000s
I20260709 13:05:57.056747 14800 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:57.058833 14884 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:57.060590  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2398) took 1239 ms. Trace:
I20260709 13:05:57.078624 14773 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.083841  9195 rpcz_store.cc:276] 0709 13:05:55.822690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.048627 (+225937us) service_pool.cc:224] Handling call
0709 13:05:56.048930 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.078418 (+ 29488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.060321 (+981903us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b54000a8054. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:57.060507 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62970112}
I20260709 13:05:57.094565 14690 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:57.094986 14662 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:57.095628 14662 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.098351 14662 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:57.099174  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:57.099435 14773 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:57.099830  9268 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:05:57.100304  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:57.100906  9120 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:57.106845  9193 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2402) took 1030 ms. Trace:
I20260709 13:05:57.107252  9193 rpcz_store.cc:276] 0709 13:05:56.076677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.117195 (+ 40518us) service_pool.cc:224] Handling call
0709 13:05:57.088218 (+971023us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54000a6754. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:05:57.106787 (+ 18569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":930455296}
I20260709 13:05:57.109547  9268 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:57.110817  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:05:57.050606 14856 consensus_meta.cc:287] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.745s	user 0.006s	sys 0.005s
W20260709 13:05:57.077498 14625 consensus_meta.cc:287] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.521s	user 0.005s	sys 0.005s
I20260709 13:05:57.080154  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:57.081048  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:57.117867  9122 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:57.120875  9119 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:57.122303  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:57.124104  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:57.125195  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:57.125680  9859 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:05:57.125946  9855 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:57.128196  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:57.132606  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:57.133132  9264 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:05:57.134507  9271 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:05:57.134100  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:57.135234  9272 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:57.134792  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:57.142693  9267 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:57.144282  9264 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:57.145445  9272 raft_consensus.cc:1275] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:05:57.129484  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:57.146132  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:57.147295  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:57.147614  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 3139) took 1114 ms. Trace:
I20260709 13:05:57.149055  9855 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:57.148072  9270 rpcz_store.cc:276] 0709 13:05:56.032899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.033359 (+   460us) service_pool.cc:224] Handling call
0709 13:05:56.033761 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:56.043176 (+  9415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:56.043202 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:56.046954 (+  3752us) log.cc:844] Serialized 26 byte log entry
0709 13:05:56.607838 (+560884us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b1800303280. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f765628c4bc 00007f76535da48b 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f76562840d1
0709 13:05:56.608026 (+   188us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:56.608067 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:56.608139 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:57.055144 (+447005us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:57.147239 (+ 92095us) raft_consensus.cc:1685] finished
0709 13:05:57.147266 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.147413 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8612,"raft.queue_time_us":9492,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":3930,"spinlock_wait_cycles":14279680,"thread_start_us":288244,"threads_started":1,"wal-append.queue_time_us":286135}
I20260709 13:05:57.119161 14773 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:57.152175 14791 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:05:57.153789 14878 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:57.153893  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:57.128330 14942 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.106280 14820 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.125576 14856 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:57.161868  9120 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:57.167263  9089 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:57.168439  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:57.136930 14836 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:57.175837  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:57.137133 14740 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:57.191547  9238 leader_election.cc:400] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 15. Current term is 16.
I20260709 13:05:57.194321  9238 leader_election.cc:403] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:05:57.192653  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2390) took 1397 ms. Trace:
I20260709 13:05:57.195922  9194 rpcz_store.cc:276] 0709 13:05:55.794939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.894602 (+ 99663us) service_pool.cc:224] Handling call
0709 13:05:55.894935 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:55.965961 (+ 71026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.192384 (+1226423us) spinlock_profiling.cc:243] Waited 60.5 ms on lock 0x7b5400218254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:57.192585 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139121408}
I20260709 13:05:57.198647  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:57.198864  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:57.199291  9192 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:57.199990  9197 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:57.182677 14625 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:57.209115  9117 raft_consensus.cc:1240] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:57.194275 14633 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:57.183329 14773 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:57.213011 14633 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:05:57.213749 14773 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.204020 14740 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.172869 14607 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.190836 14758 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:57.203385 14682 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 15. Current term is 16.
I20260709 13:05:57.203122 14839 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:05:57.203188 14690 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:05:57.242125 14690 consensus_queue.cc:1243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 94504d29c6744f38bb06943479df9466 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:05:57.244663 14794 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:57.245301 14856 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:57.177549 14942 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:57.226473 14607 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.248694  9268 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:57.254338  9122 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12. Ops: []
I20260709 13:05:57.258925  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:57.260030  9122 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:57.262787 14607 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:57.266017  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:57.252149 14740 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:57.271195 14740 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.271719 14740 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.273694 14928 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:57.274153 14928 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:57.274227 14740 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:57.274607 14928 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:57.278505 14740 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:57.279174 14740 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.256563 14927 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.279978  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:57.280179  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.280216 14927 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.270591 14689 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:05:57.282487 14927 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:57.259881 14820 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:57.282895 14690 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:05:57.283283 14820 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:57.283474 14690 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:57.283866 14820 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.284725 14690 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:57.285907 14820 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:57.286244 14740 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:57.286571 14690 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:57.289011  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:57.292666  9270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45222
I20260709 13:05:57.293246  9270 raft_consensus.cc:606] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Received request to transfer leadership
I20260709 13:05:57.293558  9270 raft_consensus.cc:614] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:05:57.318485  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1215) took 1003 ms. Trace:
W20260709 13:05:57.320305 14788 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
W20260709 13:05:57.331833 14832 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.015s	sys 0.016s
W20260709 13:05:57.332414 14832 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.015s	sys 0.017s
I20260709 13:05:57.318984  9265 rpcz_store.cc:276] 0709 13:05:56.315023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.315888 (+   865us) service_pool.cc:224] Handling call
0709 13:05:56.316157 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:56.316357 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:05:56.447594 (+131237us) log.cc:844] Serialized 14 byte log entry
0709 13:05:56.449479 (+  1885us) log.cc:844] Serialized 14 byte log entry
0709 13:05:57.317758 (+868279us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b180045e260. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f7653571c0a 00007f765ccb1219 00007f765629ad70 00007f765628e293 00007f7656283cea 00007f765628423a 00007f76566eea2a 00007f76566fe0bf 00007f76566fe038
0709 13:05:57.318032 (+   274us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:57.318105 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:57.318167 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.318258 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:57.318412 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7808,"prepare.queue_time_us":212117,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":11838,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":2851,"spinlock_wait_cycles":5796096,"thread_start_us":127495,"threads_started":1}
I20260709 13:05:57.348868  9856 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:05:57.349275  9856 pending_rounds.cc:85] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 9
I20260709 13:05:57.349504  9856 pending_rounds.cc:107] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:05:57.349728  9856 raft_consensus.cc:2889] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:57.351974 14855 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:05:57.360069  9856 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:57.363302 14376 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:05:57.372975 14844 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:57.373443 14844 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:57.374257 14844 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.375356 14844 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:56.930341  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:05:57.402081  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:57.412015  9263 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:57.416283  9267 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:05:57.420933  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:57.421097  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:57.421820  9265 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:05:57.402210 14740 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:57.408310 14927 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.424049 14927 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:57.428624  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43780 (request call id 2713) took 2258 ms. Trace:
W20260709 13:05:57.435105  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.439819 14835 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.440553 14835 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.443316 14835 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:57.465246 14960 log.cc:927] Time spent T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.306s	user 0.003s	sys 0.000s
I20260709 13:05:57.471366  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:57.472957  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:57.429149  9119 rpcz_store.cc:276] 0709 13:05:55.169906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:55.171195 (+  1289us) service_pool.cc:224] Handling call
0709 13:05:55.171517 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.121281 (+1949764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.428303 (+307022us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b5400255b54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:57.428520 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4482387712}
I20260709 13:05:57.478281  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:57.482005  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:57.488129  9121 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 20.
I20260709 13:05:57.484159  9189 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:57.489904  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:05:57.435675  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.496918  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:57.511813  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:57.521225  9826 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:05:57.518929  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:57.527616  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 335286us
I20260709 13:05:57.528735 13683 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:05:57.512058 14835 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:05:57.422883 14928 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:57.571139  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 395366us
I20260709 13:05:57.479791 14740 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:57.530800 14815 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:57.572623  9161 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:57.523277 14755 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:57.575387 14851 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.998s	user 0.019s	sys 0.010s
I20260709 13:05:57.518395 14720 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:05:57.547482 14835 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:57.586707  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:57.514631 14689 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:05:57.523638 14376 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:05:57.605140  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:57.596138 14851 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.019s	sys 0.010s
W20260709 13:05:57.611299  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 179312us
I20260709 13:05:57.592918 14682 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:57.554994 14931 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:05:57.618160  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:57.602185 14835 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:05:57.603444 14496 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:05:57.640280  9194 pending_rounds.cc:85] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 10
I20260709 13:05:57.628219  9195 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:05:57.640681  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:57.642084  9117 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:57.642799  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:57.635311  9122 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:57.645268  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:57.637104  9189 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:57.646420  9235 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:57.652176  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:57.653975  9197 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:05:57.662482  9193 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:57.643640  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:57.600147 14755 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.619913 14863 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:57.625662 14820 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:57.640605  9194 pending_rounds.cc:107] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 16.11
I20260709 13:05:57.684136  9194 raft_consensus.cc:2917] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Aborting config change with OpId 16.11: Aborted: Op aborted by new leader
I20260709 13:05:57.684659  9194 raft_consensus.cc:2889] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:05:57.692260  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.668337 14839 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:05:57.726055  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 80472.1us
I20260709 13:05:57.673280 14884 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:57.626644 14682 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.650347 14788 consensus_queue.cc:1059] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:05:57.731963  9121 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:05:57.614828 14915 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.005s	sys 0.017s
W20260709 13:05:57.731169 14851 raft_consensus.cc:670] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:05:57.733079 14927 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:57.686596 14863 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:57.733875 14839 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:57.733708 14496 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:05:57.755175 14915 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.006s	sys 0.017s
W20260709 13:05:57.871765  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 361490us
W20260709 13:05:57.970917  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 331323us
I20260709 13:05:57.971987  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:57.974081 13683 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:57.975528 13683 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:05:57.976017 13683 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:57.975805 14800 consensus_queue.cc:1048] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:57.977330 13683 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:57.978571 13683 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:57.981705 14682 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:57.984282  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:57.985375  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:57.984736 14607 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:57.985910 14607 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.992518 14607 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:57.994204 14496 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:57.996889 14607 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:58.005052 14788 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:58.050905  9856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.039s	sys 0.089s
W20260709 13:05:58.051450  9856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.039s	sys 0.089s
W20260709 13:05:58.068709  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 44480.3us
I20260709 13:05:58.017076 14839 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:58.060822 14752 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.064s	sys 0.074s
W20260709 13:05:58.070066 14821 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.130s	user 0.020s	sys 0.047s
I20260709 13:05:58.124102 14376 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:58.125293 14966 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:58.126448 14968 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.500s	user 0.001s	sys 0.000s
I20260709 13:05:58.126389 14915 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:58.127089 14971 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.439s	user 0.001s	sys 0.001s
W20260709 13:05:58.128935  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45405.6us
W20260709 13:05:58.114850 14752 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.065s	sys 0.074s
W20260709 13:05:58.131258 14821 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.021s	sys 0.047s
W20260709 13:05:58.146049  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 473391us
W20260709 13:05:58.139372  9826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.027s	sys 0.076s
I20260709 13:05:58.114426 14851 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.045230 14788 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:58.157248  9826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.027s	sys 0.078s
W20260709 13:05:58.179675  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 658498us
I20260709 13:05:58.067549 14607 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:58.176062  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.157954  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:05:58.159574 14964 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.510s	user 0.000s	sys 0.001s
W20260709 13:05:58.187976  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.188674 14863 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:05:58.202376 14972 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:05:58.202701  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:58.202929 14607 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:58.206871 14607 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.136448 14376 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.207671 14607 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.209492 14376 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.140445 14966 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.189338 14815 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.144363 14915 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:58.213758  9828 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.017s	sys 0.016s
I20260709 13:05:58.248234  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:58.253017  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:58.182026 14851 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.274412  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1603) took 1119 ms. Trace:
W20260709 13:05:58.282542  9828 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.017s	sys 0.020s
I20260709 13:05:58.292515  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:58.292841  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:58.214398 14914 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:05:58.283033  9855 rpcz_store.cc:276] 0709 13:05:57.155405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.159526 (+  4121us) service_pool.cc:224] Handling call
0709 13:05:57.159932 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.160368 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.160395 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:57.160977 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.514623 (+353646us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:57.514660 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.514725 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.274041 (+759316us) raft_consensus.cc:1685] finished
0709 13:05:58.274076 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.274326 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":353124,"threads_started":1,"wal-append.queue_time_us":449088}
I20260709 13:05:58.296679  9190 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:05:58.194430 14931 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:05:58.215299 14791 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:58.244378 14863 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.306388 14863 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.308831 14863 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.196923 14855 consensus_queue.cc:1059] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 13:05:58.217830 14803 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:58.252489 14869 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.005s	sys 0.017s
I20260709 13:05:58.505352  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:58.253587 14376 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:58.506395  9861 raft_consensus.cc:2393] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:58.493251  9859 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 18 LEARNER]: Advancing to term 19
I20260709 13:05:58.500398  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:58.252497 14815 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.297395 14758 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:58.300248 14689 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.507892  9855 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:05:58.509449  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:58.515448  9860 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:05:58.514222  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:58.428295 14863 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:58.516968  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:58.516666  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:58.518888  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:58.520414  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:58.516779  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:58.524111  9860 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
W20260709 13:05:58.524886  9162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.031s	sys 0.038s
I20260709 13:05:58.525198  9861 raft_consensus.cc:2393] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:05:58.525280  9162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.031s	sys 0.038s
W20260709 13:05:58.505363 14869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.009s	sys 0.018s
W20260709 13:05:58.526031  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 550837us
I20260709 13:05:58.547365  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:58.556553  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:58.517594 14825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.018s	sys 0.036s
I20260709 13:05:58.505026 14803 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.567293 14966 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.567914 14630 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:58.568889 14974 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.559s	user 0.002s	sys 0.000s
I20260709 13:05:58.570150 14803 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:58.571205 14983 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.390s	user 0.000s	sys 0.001s
I20260709 13:05:58.571988 14607 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:58.572679 14607 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:05:58.573478 14975 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.561s	user 0.000s	sys 0.001s
W20260709 13:05:58.575071 14835 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.924s	user 0.005s	sys 0.003s
I20260709 13:05:58.575269  9854 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1683) took 1083 ms. Trace:
W20260709 13:05:58.556829 14979 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 13:05:58.575666 14815 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:58.575492 14835 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.576750 14815 consensus_queue.cc:1048] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:05:58.577325  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1611) took 1200 ms. Trace:
I20260709 13:05:58.578833  9856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1610) took 1215 ms. Trace:
I20260709 13:05:58.577795  9853 rpcz_store.cc:276] 0709 13:05:57.377284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.377410 (+   126us) service_pool.cc:224] Handling call
0709 13:05:57.377755 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:57.380229 (+  2474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.380257 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:57.380964 (+   707us) log.cc:844] Serialized 52 byte log entry
0709 13:05:57.604809 (+223845us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:05:57.604878 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.604966 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.572834 (+967868us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:58.577001 (+  4167us) raft_consensus.cc:1685] finished
0709 13:05:58.577023 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.577260 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13889,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":114604416,"thread_start_us":223438,"threads_started":1,"wal-append.queue_time_us":631332}
W20260709 13:05:58.579236 14844 consensus_meta.cc:287] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 1.204s	user 0.012s	sys 0.000s
I20260709 13:05:58.573199 14607 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.580425  9859 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 LEARNER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:05:58.544420 14931 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:58.610817  9855 raft_consensus.cc:2410] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:05:58.515789 14758 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.524663 14689 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:58.513269 14376 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:58.579173  9856 rpcz_store.cc:276] 0709 13:05:57.363906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.364140 (+   234us) service_pool.cc:224] Handling call
0709 13:05:57.364523 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.377647 (+ 13124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.377676 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:57.383477 (+  5801us) log.cc:844] Serialized 26 byte log entry
0709 13:05:58.051891 (+668414us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:58.051952 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.052050 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.578606 (+526556us) raft_consensus.cc:1685] finished
0709 13:05:58.578625 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.578783 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":70,"thread_start_us":640120,"threads_started":1,"wal-append.queue_time_us":898904}
I20260709 13:05:58.583181  9854 rpcz_store.cc:276] 0709 13:05:57.492043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.494804 (+  2761us) service_pool.cc:224] Handling call
0709 13:05:57.495211 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.508758 (+ 13547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.508789 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:57.509277 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.966424 (+457147us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:05:57.966488 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.966594 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.570589 (+603995us) spinlock_profiling.cc:243] Waited 6.27 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:58.574874 (+  4285us) raft_consensus.cc:1685] finished
0709 13:05:58.574902 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.575191 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13083,"raft.queue_time_us":13275,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5508,"spinlock_wait_cycles":14429056,"thread_start_us":441584,"threads_started":1,"wal-append.queue_time_us":606038}
W20260709 13:05:58.589529  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:58.587730  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:58.578483  9862 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:05:58.557437 14982 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:58.578094 14835 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:58.564975 14825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.018s	sys 0.037s
I20260709 13:05:58.533790 14863 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.626652 14869 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:58.631687 14835 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:58.634599 14788 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.626386 14982 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.638715 14984 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:58.642706 14662 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.012s	sys 0.050s
W20260709 13:05:58.642896 14980 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
I20260709 13:05:58.640708 14915 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.616859 14931 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
W20260709 13:05:58.658366 14662 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.013s	sys 0.050s
I20260709 13:05:58.659348 14851 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:58.645653 14869 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.648522 14835 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:58.731568  9086 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.005s	sys 0.026s
W20260709 13:05:58.780910  9086 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.006s	sys 0.026s
I20260709 13:05:58.692893 14931 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:58.733946  9089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.010s	sys 0.006s
W20260709 13:05:58.781992  9089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.010s	sys 0.007s
I20260709 13:05:58.782454 14931 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:58.784209 14928 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:58.784451  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 626559us
I20260709 13:05:58.783811 14931 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.785472 14863 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.785573 14928 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:58.734790  9087 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.004s	sys 0.031s
W20260709 13:05:58.786827  9087 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.004s	sys 0.031s
I20260709 13:05:58.786685 14928 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:58.744251  9164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.010s	sys 0.025s
W20260709 13:05:58.787916  9164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.011s	sys 0.025s
I20260709 13:05:58.788451 14931 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.685187 14942 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:58.745486  9163 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.004s	sys 0.007s
W20260709 13:05:58.789726  9163 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.004s	sys 0.007s
I20260709 13:05:58.790017 14869 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:05:58.733848  9069 auto_rebalancer.cc:893] Removing replace marker failed: Replica 1efabcb1befc400aadf101e8276fc1aa is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:05:58.791301  9069 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:05:58.744416  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 162916us
W20260709 13:05:58.791837  9069 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
W20260709 13:05:58.756259  9238 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.005s	sys 0.013s
W20260709 13:05:58.793540  9238 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.005s	sys 0.013s
W20260709 13:05:58.756476  9265 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.011s	sys 0.003s
W20260709 13:05:58.795982  9265 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.012s	sys 0.003s
I20260709 13:05:58.797070 14928 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.797925 14928 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:05:58.765780  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 263205us
W20260709 13:05:58.767444  9826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.007s	sys 0.013s
W20260709 13:05:58.800241  9826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.008s	sys 0.013s
I20260709 13:05:58.769224 14607 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:58.774748 14689 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:58.777331 14758 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.763761  9264 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2379) took 1357 ms. Trace:
I20260709 13:05:58.795409  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:58.766326 14376 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:58.696092 14982 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:58.805676 13683 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.806315 14982 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:05:58.806522  9828 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:58.661610 14984 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:05:58.695281 14991 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 13:05:58.703467 15001 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:58.817305 15001 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.802253  9264 rpcz_store.cc:276] 0709 13:05:57.406428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.406555 (+   127us) service_pool.cc:224] Handling call
0709 13:05:57.406920 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.407345 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.407386 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:57.412415 (+  5029us) log.cc:844] Serialized 26 byte log entry
0709 13:05:57.678229 (+265814us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:57.678297 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:57.678397 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:58.763073 (+1084676us) raft_consensus.cc:1685] finished
0709 13:05:58.763118 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:58.763525 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1852,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10694,"thread_start_us":253431,"threads_started":1,"wal-append.queue_time_us":763146}
I20260709 13:05:58.804360 14836 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.806631 13683 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.820533 14740 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:58.822723 14835 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:58.822784 13683 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.801812 14825 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:58.830804  9118 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:05:58.830307  9271 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:05:58.830201  9270 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
W20260709 13:05:58.842586  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 47446.7us
I20260709 13:05:58.844781  9270 pending_rounds.cc:85] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 11
I20260709 13:05:58.861896  9270 pending_rounds.cc:107] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:05:58.862381  9270 raft_consensus.cc:2889] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:58.823704 14766 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:58.870368  9122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.009s	sys 0.005s
W20260709 13:05:58.871022  9122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.010s	sys 0.005s
I20260709 13:05:58.877300 14984 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:58.878913  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:58.880340  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 74129.5us
I20260709 13:05:58.882282 14928 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:58.882907 14758 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.830809 14855 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:58.896528 14820 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:58.829942 14752 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.074s
W20260709 13:05:58.908085  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 1.25585e+06us
I20260709 13:05:58.910907 14740 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:58.912562 15001 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.875277 14376 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.824455 14915 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:05:58.820536 14836 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.824172 14825 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:58.857805 14982 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:05:58.924578 14766 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.925325 14914 raft_consensus.cc:3055] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:05:58.942718 14914 raft_consensus.cc:740] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:58.943934 14766 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.944500 14914 consensus_queue.cc:260] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.946218 14914 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:05:58.948958  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 70347.6us
I20260709 13:05:58.950271 14984 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:58.960222 14758 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:58.963316 14758 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:05:58.964959 14928 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:58.965054 14662 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:58.964044 14826 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:58.937032 14820 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:58.936333 14740 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:58.985918  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:58.983734 14740 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:58.977085 14982 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:59.004485 14826 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.006916 14826 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.008195 14815 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:59.009719 14815 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:05:59.010179 14815 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:59.011159 14996 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:59.012118 14944 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.012475 15000 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:59.012283 14815 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.012753 14376 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.013383 15000 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.014153 14815 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:05:59.015347 14376 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.016315 14942 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:59.017071 14942 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.017514 14825 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.022392  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 2891) took 1035 ms. Trace:
I20260709 13:05:59.011096 14426 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:05:59.022013  9194 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2423) took 1382 ms. Trace:
I20260709 13:05:59.019630 14942 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:58.999681 14931 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:59.030614  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:59.039906 14996 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.044165  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.045079 14928 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.047334 14832 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:59.026077  9122 rpcz_store.cc:276] 0709 13:05:57.986905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.987064 (+   159us) service_pool.cc:224] Handling call
0709 13:05:57.987344 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:57.987539 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:05:58.239623 (+252084us) log.cc:844] Serialized 14 byte log entry
0709 13:05:58.928662 (+689039us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:05:58.938353 (+  9691us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:05:58.940938 (+  2585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.981656 (+ 40718us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.022334 (+ 40678us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54105,"prepare.queue_time_us":639692,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":2442,"raft.queue_time_us":57222,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":47012,"thread_start_us":873761,"threads_started":2,"wal-append.queue_time_us":596939}
I20260709 13:05:59.048386 14836 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.048072 14832 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.052524 14832 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.050203  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:59.055577  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 69826.8us
I20260709 13:05:59.029412 14766 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:59.056908 14766 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.059439 14766 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.028051 14376 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:05:59.078184  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.075552 14944 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.081612 14740 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.082862 14690 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:59.089814 14690 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.092411 14690 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.084285 14944 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.084861 15000 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.085150 14795 raft_consensus.cc:3055] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:05:59.093950 14795 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:05:59.072647 14855 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.095041  9828 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:59.096750 14740 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.096881 14855 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.075471  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.073598 14928 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.088711 15001 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:59.100185 15001 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.102713 15001 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.103276 15001 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:05:59.099121 14928 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.106355 14928 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.108419 14982 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.111334 14844 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:05:59.112954  9828 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:05:59.100446  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 125426us
W20260709 13:05:59.109151 14970 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.096s	user 0.003s	sys 0.000s
I20260709 13:05:59.116870 14931 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:05:59.118108 14987 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.488s	user 0.000s	sys 0.003s
I20260709 13:05:59.031068  9194 rpcz_store.cc:276] 0709 13:05:57.639571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.639719 (+   148us) service_pool.cc:224] Handling call
0709 13:05:57.640081 (+   362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:05:57.993693 (+353612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.993750 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:05:57.994703 (+   953us) log.cc:844] Serialized 52 byte log entry
0709 13:05:58.239956 (+245253us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:05:58.296109 (+ 56153us) log.cc:844] Serialized 14 byte log entry
0709 13:05:58.873649 (+577540us) spinlock_profiling.cc:243] Waited 946 us on lock 0x7b18003e4760. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f7653571c0a 00007f765ccb1219 00007f76535f9a40 00007f76535f44e4 00007f76535f4455 00007f765a0dc3c7 00007f76566ee9a7 00007f76566fe0bf 00007f76566fe038
0709 13:05:58.936303 (+ 62654us) log.cc:844] Serialized 14 byte log entry
0709 13:05:58.950575 (+ 14272us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b18003e4760. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:05:58.997515 (+ 46940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.021390 (+ 23875us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.021450 (+    60us) raft_consensus.cc:1685] finished
0709 13:05:59.021483 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.021723 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":549012,"prepare.queue_time_us":635175,"prepare.run_cpu_time_us":1801,"prepare.run_wall_time_us":9063,"raft.queue_time_us":296923,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":162,"spinlock_wait_cycles":84047232,"thread_start_us":244805,"threads_started":1,"wal-append.queue_time_us":621825}
I20260709 13:05:59.121837 14755 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.122215 14376 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.126390  9192 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2422) took 1487 ms. Trace:
I20260709 13:05:59.126863  9192 rpcz_store.cc:276] 0709 13:05:57.639291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:57.639647 (+   356us) service_pool.cc:224] Handling call
0709 13:05:57.668831 (+ 29184us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:57.736760 (+ 67929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:57.736811 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:57.740348 (+  3537us) log.cc:844] Serialized 26 byte log entry
0709 13:05:58.214832 (+474484us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:05:58.214896 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:58.216197 (+  1301us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:05:59.117245 (+901048us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:05:59.125823 (+  8578us) raft_consensus.cc:1685] finished
0709 13:05:59.125849 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:05:59.126212 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67410,"raft.queue_time_us":142407,"spinlock_wait_cycles":43460224,"thread_start_us":247824,"threads_started":1,"wal-append.queue_time_us":635993}
I20260709 13:05:59.069726 14662 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.136802 14662 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.043592 14996 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.139544  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.140436  9117 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:59.147603  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.148685  9119 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:59.149371  9117 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 13:05:59.143500  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.151402  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.156062 14931 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.156510  9121 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
W20260709 13:05:59.159199 14978 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.661s	user 0.000s	sys 0.004s
I20260709 13:05:59.147861  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:59.161835  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46128.1us
I20260709 13:05:59.166225  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:59.167431 14908 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.168248 14908 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.167486 14690 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.170461 14690 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.172037 14965 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.172577 14908 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.161937  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 1052) took 2805 ms. Trace:
W20260709 13:05:59.196300  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.174667 14802 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.173810  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.172641 14965 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.176458 14690 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:59.177634 14825 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.179912  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.177903 14809 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:05:59.195865  9118 rpcz_store.cc:276] 0709 13:05:56.356802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:56.375468 (+ 18666us) service_pool.cc:224] Handling call
0709 13:05:56.395511 (+ 20043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:05:58.865836 (+2470325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:05:59.161649 (+295813us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b5400120a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:05:59.161867 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5599309056}
I20260709 13:05:59.175393 14985 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:59.189169  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 178653us
W20260709 13:05:59.193053 15007 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.148s	user 0.000s	sys 0.010s
I20260709 13:05:59.189808 14844 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:05:59.205440 14820 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.228410 14802 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.229557  9268 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:59.229890  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.277750  9197 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:05:59.230387  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.230509 14826 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:59.195534 14740 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:05:59.236131  9272 raft_consensus.cc:2410] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:05:59.236466  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.196707 14795 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.216132  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.245573  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.246393  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:59.248396  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.248282  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.250484 14800 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.251189 14662 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:59.247370  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.252008  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.303349  9194 raft_consensus.cc:2393] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 16: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:59.252202  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.252856  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.253161 14950 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:59.254134  9191 raft_consensus.cc:1240] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: []
I20260709 13:05:59.254717 14627 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:59.256407 14792 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.257741 14791 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:05:59.258442  9855 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:05:59.259112  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.259034 14836 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:59.261439  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.266713 14376 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.317821 14792 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.320513 14792 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.312255  9855 pending_rounds.cc:85] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 11
I20260709 13:05:59.321173  9855 pending_rounds.cc:107] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:05:59.321449  9855 raft_consensus.cc:2889] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:59.267069 14965 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.323117  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:59.229614  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 63581.5us
I20260709 13:05:59.324607  9853 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:05:59.225124 14809 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:05:59.325265 14795 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:05:59.325600  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:59.326409 14809 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.329639 14802 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.269205  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.331488  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.332544  9860 raft_consensus.cc:2410] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 13:05:59.269243  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 156475us
I20260709 13:05:59.271732 14825 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.334124  9828 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:59.272529  9118 raft_consensus.cc:1240] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 7. Current term is 8. Ops: []
I20260709 13:05:59.288702  9116 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:59.289835  9270 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:05:59.327324  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:59.300935  9271 raft_consensus.cc:2410] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:05:59.307003  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.307307  9862 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:59.307734 14662 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.308655 14800 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.308513 14950 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.309458 14826 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.309700 14627 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.310324 14821 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:05:59.314370 14736 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.314807 14985 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.315946  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.343968  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.345631  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.336455 14740 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:59.368578  9855 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:05:59.370718  9859 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:05:59.362177 14809 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:59.381588  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.382627  9268 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:59.385154  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.370987 14844 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:59.386806  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.387632  9162 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:59.386232  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:59.379210 14825 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.404247  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:59.405030  9119 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:05:59.401441 14884 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.321s	user 0.017s	sys 0.008s
I20260709 13:05:59.408208  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:59.408821 14884 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.393s	user 0.017s	sys 0.009s
W20260709 13:05:59.402202  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 68227.9us
I20260709 13:05:59.411221  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:05:59.412781 14981 consensus_queue.cc:1059] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:05:59.413947 14981 raft_consensus.cc:3055] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:05:59.414295 14981 raft_consensus.cc:740] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:05:59.415796 14981 consensus_queue.cc:260] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.417371 14981 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:59.403036 14950 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.419912 14690 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:59.420635 14690 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.421165 14690 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:59.422485 14662 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.367956 14426 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
W20260709 13:05:59.424117  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 106790us
I20260709 13:05:59.388794 14844 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:05:59.439291  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 70096.8us
I20260709 13:05:59.440227  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:59.430996 14662 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:05:59.395435 14736 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.425493 14914 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.431929 14844 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:59.436381 14982 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:59.470220  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.468729  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:05:59.471454  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 60385.5us
I20260709 13:05:59.473666  9117 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:59.466967 14950 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:59.474556 14950 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:59.474944 14950 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:59.477741 14985 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.451074 14662 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.465770 14836 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:59.491248  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:59.494760  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.481457 14820 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.467092 14800 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.500589  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.513758  9117 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:59.488611 14825 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:05:59.519061  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:05:59.519827  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.481644 14965 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:05:59.493172 14836 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:59.493216 14662 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:59.496784 14791 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:05:59.492755 14915 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:59.520928 14802 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:59.501044 14815 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:59.549856  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.520638 14825 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:59.520182 14826 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.521263 14982 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:59.536099 14836 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:59.592473 14982 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.559927  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.595925 14982 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:59.561334  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.596827  9115 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:59.597728  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.561770 14627 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.598481  9189 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:05:59.562415  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.562631 14815 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.564245  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.564395  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:59.565441 15023 log.cc:927] Time spent T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:05:59.565979  9198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:05:59.617018  9198 raft_consensus.cc:606] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:59.617431  9198 raft_consensus.cc:614] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:05:59.567242 15025 log.cc:927] Time spent T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:05:59.569041  9268 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:59.572225  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.573125  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.574040  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.550711 14986 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:05:59.558255 15015 log.cc:927] Time spent T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:05:59.590636 14915 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.592519 14802 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:05:59.634886  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.539422 14773 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:05:59.602982  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.501430 14914 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.622725  9116 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:05:59.601879 14825 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:05:59.625859  9114 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:05:59.623565  9118 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:59.626430  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.630328  9119 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 21.
I20260709 13:05:59.631327  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.632009  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.632095  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.634294  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.560537  9272 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:59.637789  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.640761  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.647454  9862 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:05:59.648804  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.646705  9857 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:59.645709  9195 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:59.654507  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:05:59.659917  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:05:59.661377  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.663605 14376 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.664247 14376 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.665753 14825 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.666157 14986 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.667093  9189 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 19: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:59.667273 14720 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:59.667972 14720 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.668942 14986 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:59.670248 14720 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.671252 14802 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.672624  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.672930 14825 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.673699  9192 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 17: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:05:59.673632  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.684166  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.689280 14690 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.690413  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.690891 14915 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:59.693157  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 346633us
I20260709 13:05:59.693531  9121 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:05:59.694228  9164 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:59.695297 14627 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:59.695724 14627 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.696099 14627 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:59.697654 14815 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.698953  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.699641  9116 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:59.641589 14736 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.700120  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.700894  9120 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:05:59.702277  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.689620  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.703137  9114 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:05:59.691488 14690 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.708114  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:59.708482  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.708871  9272 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:59.710654  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.716238  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.718111  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:59.666687 14376 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.719446 14884 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:59.720013 14740 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:05:59.721064  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:59.722277  9235 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:59.723307  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.724148  9265 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:05:59.725155  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.730284 14627 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.732061 14984 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.746711  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:59.747236 14984 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.732817 14792 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:05:59.736249  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.693186 14914 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.737216  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.737617  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.720023 14884 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.740422  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.741325  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.741492  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.742719 15010 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:05:59.747712 14950 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.762624 14950 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.749626 14984 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.749713 14825 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:05:59.763926  9190 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:05:59.749876 14792 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.766687 14792 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:05:59.743026 15021 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.296s	user 0.003s	sys 0.000s
I20260709 13:05:59.744464 14986 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:05:59.768021 14986 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 21 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:05:59.769541 14914 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.769198 14986 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.770135 14914 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.771919 14690 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:05:59.772403  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.753233  9189 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:05:59.774519  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.775321  9861 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:05:59.774590  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.752966 14627 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.775722  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.755079 14884 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.753491 14836 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.757548  9264 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:59.760035 15010 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:05:59.777769  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.757524 14736 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:05:59.772526 14690 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.781010  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.781713  9862 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:05:59.772449 14914 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:05:59.782864 14690 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:05:59.778111 14836 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.785048  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.773384  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.785667  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:59.779376  9854 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 18.
I20260709 13:05:59.778002 15010 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:05:59.786996 15001 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:05:59.760097  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 64460.1us
I20260709 13:05:59.775883  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:59.788357  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:05:59.789198  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.789441  9164 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:05:59.790004  9857 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:59.790570  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.790766  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:59.791432  9861 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:59.792507  9858 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:05:59.790297  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.794085  9860 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:05:59.794916  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.795585  9853 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:59.796463  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:59.797240  9019 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:59.799923 14792 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:05:59.800390 14792 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.800748 14792 raft_consensus.cc:3060] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:05:59.802726 14825 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:05:59.802809  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.779062  9272 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:59.804114  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.803432 14825 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.790095  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.804726  9860 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:05:59.791337  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.807035  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.814850  9857 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:05:59.816586  9861 raft_consensus.cc:2468] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
W20260709 13:05:59.820607  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 60877.4us
I20260709 13:05:59.822027  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.824846  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.833434  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.834137  9190 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:59.834872  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.835179  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.804860 14825 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:59.826946  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:05:59.855839  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.864454  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.864773  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:59.802644  9019 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:05:59.865190  9194 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:05:59.870211  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:05:59.870524  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:05:59.858011  9235 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:59.872660  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:05:59.878732  9235 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:05:59.871058  9198 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:05:59.878425  9858 pending_rounds.cc:85] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 11
I20260709 13:05:59.884176  9858 pending_rounds.cc:107] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:05:59.884560  9858 raft_consensus.cc:2889] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:05:59.896265  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:05:59.900743  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:59.906359  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.880666 14884 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:05:59.909763  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 70237.1us
I20260709 13:05:59.911223  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.885375 14985 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:05:59.916267  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 86981.9us
I20260709 13:05:59.919298  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:59.926288  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.914001 14996 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:05:59.918913 14985 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:05:59.931327 14996 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.931596 14985 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:05:59.931816  9195 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:05:59.932230  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:59.933019  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:05:59.933759  9115 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:05:59.934015  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.934666  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:59.931864 14996 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:05:59.940459 14792 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.940901 14985 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:05:59.942708 14792 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:05:59.952741  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:59.914999 14884 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:59.945327  9238 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:59.953912  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.955255  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.948179  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:59.968218  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:05:59.970996  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:59.950178  9855 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:05:59.971694  9264 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:59.952101  9192 raft_consensus.cc:2368] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12.
I20260709 13:05:59.955904  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:05:59.956174  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:05:59.972985  9854 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:05:59.960294  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:05:59.964677  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.965615  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.966882  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.968207  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.974743  9114 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:05:59.933764 14720 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:05:59.953351 14627 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:59.974850  9270 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:05:59.972033  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:59.947237 14985 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:05:59.986387  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.989789  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:05:59.990840  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.992189  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.991537  9118 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:05:59.953621 14809 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:05:59.915367 14844 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.963615 15010 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.017277 14844 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.017505 15010 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:00.018352 15010 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:05:59.982439 14690 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:05:59.991400 14627 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:05:59.955361 14884 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.020740  9268 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:00.021559 14627 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:00.023077  9859 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:00.024506  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 56368.1us
I20260709 13:06:00.026181  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.026985  9194 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 18: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:00.027765  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.028829  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:00.029510  9194 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:00.030228  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.031109  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.998296  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:05:59.998970  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:05:59.994544  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:00.002065  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.004048  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:00.010094  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:05:59.972193 14815 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.010828  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.046862  9198 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:05:59.996098 14792 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:00.059484 14815 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.060014 14825 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:00.060882  9015 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36626 (request call id 361) took 20088 ms (client timeout 15000 ms). Trace:
I20260709 13:06:00.062283 14825 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:00.066176  9192 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:00.031637  9194 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:00.055552 14662 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.057685  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:00.063628  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:00.080319  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.082020  9193 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:00.082232  9264 raft_consensus.cc:2410] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:06:00.086439  9190 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:00.088595  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.089896  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:06:00.089542  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47463.6us
I20260709 13:06:00.091106  9164 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:00.098578  9854 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:06:00.099907  9189 raft_consensus.cc:2368] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 15. Current term is 16.
I20260709 13:06:00.038585  9019 catalog_manager.cc:5697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 6 to 11, leader changed from <none> to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 11 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:05:59.992250 14996 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.103049 14662 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:00.103076  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.096827 14690 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:06:00.103688  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 105472us
I20260709 13:06:00.107364  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:00.107932 14815 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.109681  9161 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:00.061498  9015 rpcz_store.cc:269] 0709 13:05:39.972597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.974989 (+  2392us) service_pool.cc:224] Handling call
0709 13:06:00.060607 (+20085618us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:05:53.051504 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:05:53.052629 (+  1125us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:05:53.052692 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:05:53.052721 (+    29us) tablet.cc:662] Decoding operations
0709 13:05:53.085091 (+ 32370us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:05:53.085202 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:05:53.085226 (+    24us) tablet.cc:685] Acquiring locks for 26 operations
0709 13:05:53.087291 (+  2065us) tablet.cc:701] Row locks acquired
0709 13:05:53.087344 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:05:53.087531 (+   187us) write_op.cc:270] Start()
0709 13:05:53.087678 (+   147us) write_op.cc:276] Timestamp: P: 1783602353087453 usec, L: 0
0709 13:05:53.087715 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:05:53.132729 (+ 45014us) log.cc:844] Serialized 10522 byte log entry
0709 13:05:54.095760 (+963031us) op_driver.cc:464] REPLICATION: finished
0709 13:05:54.454789 (+359029us) write_op.cc:301] APPLY: starting
0709 13:05:54.473805 (+ 19016us) tablet.cc:1367] starting BulkCheckPresence
0709 13:05:55.460996 (+987191us) tablet.cc:1370] finished BulkCheckPresence
0709 13:05:55.461129 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:05:55.952069 (+490940us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:05:55.952498 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=0,mrs_lookups=0
0709 13:05:55.952562 (+    64us) write_op.cc:312] APPLY: finished
0709 13:05:55.953106 (+   544us) log.cc:844] Serialized 228 byte log entry
0709 13:05:56.022815 (+ 69709us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:56.048571 (+ 25756us) write_op.cc:454] Released schema lock
0709 13:05:56.055267 (+  6696us) write_op.cc:341] FINISH: Updating metrics
0709 13:05:56.106744 (+ 51477us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:05:56.123178 (+ 16434us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":264,"mutex_wait_us":1216635,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":340513,"apply.run_cpu_time_us":68558,"apply.run_wall_time_us":1639452,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112859,"num_ops":26,"prepare.queue_time_us":300778,"prepare.run_cpu_time_us":16102,"prepare.run_wall_time_us":196578,"raft.queue_time_us":254533,"raft.run_cpu_time_us":31581,"raft.run_wall_time_us":243105,"replication_time_us":973735,"thread_start_us":783229,"threads_started":5,"wal-append.queue_time_us":138975,"wal-append.run_cpu_time_us":92886,"wal-append.run_wall_time_us":2775601}]]}
I20260709 13:06:00.096223 14773 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:00.068205 14985 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:06:00.103348  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:00.119861  9161 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:00.116063  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 58445.7us
I20260709 13:06:00.104235 14690 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.122225  9192 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:00.104276 14908 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:06:00.104323 14802 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:00.103204 14996 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:00.103916 14662 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:00.131871  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:00.127707 14802 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.147372  9192 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:00.148475  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:00.150641  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:00.121418 14773 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:00.158034 14496 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:00.166038  9190 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:06:00.152336 14802 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:00.141562 14821 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:00.201041  9189 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:00.171373 14690 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:00.177058  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52034.1us
I20260709 13:06:00.177814  9828 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:06:00.178910  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 59172.7us
W20260709 13:06:00.179491  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 57699.9us
W20260709 13:06:00.180382  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48636.1us
I20260709 13:06:00.203878  9164 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:00.130949 14825 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:00.127933 14736 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:00.214969  9164 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:00.125098 14815 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:06:00.184947 14773 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.206017 14792 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:00.225483  9828 leader_election.cc:400] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12.
I20260709 13:06:00.229574  9828 leader_election.cc:403] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:06:00.205309 14496 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:00.226507  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:00.227324  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:00.233055  9119 raft_consensus.cc:2410] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:00.233119  9121 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:00.205614 14836 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:00.201406 14821 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:00.216024 14690 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:00.238343  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:00.247663  9115 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:06:00.221673 14627 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:00.220068 14985 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:00.227892 14981 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:00.257757 14985 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:00.260111 14984 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12.
I20260709 13:06:00.236408 14795 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:00.221797 14844 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:06:00.245707 14986 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:00.282040 14985 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:00.282176  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 347549us
I20260709 13:06:00.261673 14981 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:00.279088 14965 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:00.282923 14836 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.260531 14690 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:06:00.235519 14496 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.291878 14986 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:06:00.304141  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 91798.4us
I20260709 13:06:00.300408 14981 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:00.304979  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.342322 15001 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:06:00.346676  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46370.1us
I20260709 13:06:00.308125 14986 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.349742 15019 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:00.308753 14996 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.352644  9089 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.305167 14836 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:00.348193 14915 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.385488  9115 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
W20260709 13:06:00.385308  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 499386us
I20260709 13:06:00.391799  9089 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:06:00.361780 15038 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:06:00.390432  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:00.358665 15019 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:00.349360 15001 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:00.359555 14809 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:00.366681 14884 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:00.407891  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.416316  9089 leader_election.cc:400] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 15. Current term is 16.
I20260709 13:06:00.416785  9089 leader_election.cc:403] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:00.405839 15001 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:00.342952 14821 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:00.404034 15019 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:00.389724 14996 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:00.410020 14376 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:00.407925 14809 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:00.393695 14915 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:00.429382  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 279001us
I20260709 13:06:00.435878 14376 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.436493 14376 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.438090  9019 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 6 to 8, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 8 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:00.440296 14809 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.440735  9269 raft_consensus.cc:2368] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13.
I20260709 13:06:00.444255 14914 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for earlier term 15. Current term is 16.
W20260709 13:06:00.445943 15042 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:06:00.422151 14720 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:00.430789  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:00.470506  9121 raft_consensus.cc:2468] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:06:00.459851  9019 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 4 to 7, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 7 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:00.463308 14996 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.494169  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:00.444161 14815 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:00.516877  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:00.523784  9853 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:06:00.477402 14802 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.656220  9117 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:00.473847 14931 raft_consensus.cc:2764] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:06:00.590852  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:06:00.498306  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 272965us
I20260709 13:06:00.662539  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:00.667851  9164 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:00.563452  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:00.542894  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.672650  9190 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:06:00.553436  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:00.499046 14821 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.564311  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:00.597348  9087 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.67:38807 timed out after 1.612s (SENT) [suppressed 1 similar messages]
I20260709 13:06:00.670161  9120 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 8.
I20260709 13:06:00.563021  9161 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.670053  9857 raft_consensus.cc:2393] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 12: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:00.552354  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:00.565972  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.566310  9197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:00.681730  9197 raft_consensus.cc:606] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:00.682160  9197 raft_consensus.cc:614] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:00.600674  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 210291us
I20260709 13:06:00.632158  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:00.602753  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:00.684731  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:00.592862  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:00.498270 14740 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:00.686434  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:06:00.525136  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 94485us
I20260709 13:06:00.683960  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:00.690080  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:00.688122  9238 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:00.646102 14996 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:00.690888  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:00.694350  9162 leader_election.cc:400] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13.
I20260709 13:06:00.694638  9162 leader_election.cc:403] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:06:00.691529  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 128577us
I20260709 13:06:00.695421  9114 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:00.601835 15067 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:00.619204 14884 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:00.695822  9161 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.698215  9855 raft_consensus.cc:2410] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:06:00.568109 15055 raft_consensus.cc:2764] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:06:00.697636 14800 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:00.700642 14984 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:00.701800  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:00.702427 14908 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:00.693060  9198 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:00.692452  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:00.690677 14720 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.648121 14809 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:00.711539 14627 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:00.556990 14496 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.717715 14915 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:00.690888 15053 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:00.717823  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:00.718917 14802 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:00.720170 14773 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.710242 15067 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.721066 14884 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:00.723073 15073 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:00.693830 14426 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:00.696764 14795 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:00.560111 14815 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.744586  9269 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 7.
I20260709 13:06:00.767735  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:00.712491 14984 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:00.747879 15074 raft_consensus.cc:2804] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:00.711815 14844 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:00.727443 14792 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:00.750505  9235 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.763993  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:00.778854  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:00.743454 15050 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:00.703969 14996 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.779588 15050 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:00.780098 15050 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:00.777720  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 65396.6us
I20260709 13:06:00.755177 14627 raft_consensus.cc:697] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:00.781261 14996 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:00.751621  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59231.5us
I20260709 13:06:00.781780 14627 consensus_queue.cc:237] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.783285 14914 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:00.783982  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.753881 14825 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:00.755136 14884 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.786048 14884 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:00.788282 14996 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:00.788556  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:00.788807 14996 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:00.789202 14996 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:00.780323  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 78630.1us
I20260709 13:06:00.790834  9164 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:00.749874 14981 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:00.793895  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 196579us
W20260709 13:06:00.795606  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 45215.1us
I20260709 13:06:00.774670 14795 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:00.748304 15010 consensus_queue.cc:1059] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:06:00.795305 14426 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:00.801311 14984 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.804762 14809 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.807006 14795 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:00.800153 15066 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:06:00.810410  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:00.811117  9264 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:00.800168 15049 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:00.814517 15049 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:00.814963 15049 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:00.815414 15010 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:00.815603 14981 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.816421 14981 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:00.792070 15074 raft_consensus.cc:697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:00.832516 15074 consensus_queue.cc:237] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.820302 14773 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.820026 14844 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.815938 15010 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:00.827566 15050 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.819897 15067 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.844928 15067 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:00.845577 15067 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.836345 14815 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:00.838276  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 110190us
I20260709 13:06:00.841132  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:00.842757  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55589.7us
I20260709 13:06:00.834700 14836 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.846900  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:00.858481  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 94742.8us
I20260709 13:06:00.769658 15073 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.854431 14376 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:00.842861 14426 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:00.837568 14981 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:00.868692 14981 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:00.874079  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:00.874802  9854 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:06:00.880292 14720 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:00.880546 14426 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.881839 15010 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.883317 15010 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:00.884575  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:00.893270  9118 raft_consensus.cc:1240] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 16. Current term is 17. Ops: []
W20260709 13:06:00.893393  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 99993.9us
I20260709 13:06:00.891700  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.909222  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:00.909986  9854 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
W20260709 13:06:00.917124  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 57036.1us
I20260709 13:06:00.917524  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.917379  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:00.923465  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:00.925379  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:00.927954  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:00.928802  9118 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 13: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:00.938736 15050 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:00.939514  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:00.939618  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:06:00.926227 14985 consensus_meta.cc:287] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.621s	user 0.007s	sys 0.005s
I20260709 13:06:00.942844  9116 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:06:00.952347  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:00.949913  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:00.958702  9192 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:00.960166  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 75435.6us
I20260709 13:06:00.937182 15083 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:00.946405  9089 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:00.963593  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:00.965584  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:00.967370  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:00.968827  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:00.969403  9197 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:00.970098  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.973896  9120 raft_consensus.cc:2393] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:00.975307  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:00.975951  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:00.953430 14826 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:06:00.981171  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:00.985566  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 50128.4us
I20260709 13:06:00.987769  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:00.989845 15084 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:00.943297 14815 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:06:00.987347 14844 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.946298 15085 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:00.994676 15049 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:00.998519  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:00.998399  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:01.000397  9116 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:01.001875  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:01.017526  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.020639  9194 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
W20260709 13:06:01.022732  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 87268.1us
I20260709 13:06:00.996495 14792 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13.
I20260709 13:06:01.017450 14376 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:00.947580 14985 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:00.951445 14773 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:06:00.986274 14826 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.068694  9190 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:01.069063  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:00.997875 14720 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:01.019661 15085 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:01.071321  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 67902.1us
I20260709 13:06:01.075109  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:01.068843  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:01.071416  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48524.6us
W20260709 13:06:01.082540  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 94866us
I20260709 13:06:01.084929  9272 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:01.110857  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:01.075922 14826 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:01.073230 15067 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:01.080359  9193 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:06:00.980525 15083 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.079465 15053 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:01.077054 14844 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:01.123847  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 147973us
I20260709 13:06:01.078164 14720 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.158317 14720 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:01.077687 15010 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:01.162918 14981 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.077450 14496 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.134176  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.165308  9193 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:01.079676 15085 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:01.162863 15010 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:01.174319 15010 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:01.175025  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.175261  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:01.175951  9265 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:01.176829  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:01.177642  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:01.178220 15095 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:01.137118 14844 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.179874 14844 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:01.180267  9195 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2436) took 1224 ms. Trace:
I20260709 13:06:01.180771  9195 rpcz_store.cc:276] 0709 13:05:59.956308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.096455 (+140147us) service_pool.cc:224] Handling call
0709 13:06:00.106196 (+  9741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.213089 (+106893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:00.213145 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:00.378720 (+165575us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b2c000fc1b8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:06:00.379251 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:06:00.600072 (+220821us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:00.665351 (+ 65279us) log.cc:844] Serialized 14 byte log entry
0709 13:06:00.684997 (+ 19646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.941926 (+256929us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b180040a320. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f7522 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:06:01.179860 (+237934us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b5400218254. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:01.179925 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:01.179986 (+    61us) raft_consensus.cc:1685] finished
0709 13:06:01.180005 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.180198 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95548,"prepare.queue_time_us":61233,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":77048,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":13172,"spinlock_wait_cycles":264545920,"thread_start_us":220358,"threads_started":1,"wal-append.queue_time_us":310873}
I20260709 13:06:01.186481  9120 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:01.190039  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:01.154939 15083 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:01.157197 14986 consensus_meta.cc:287] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.771s	user 0.012s	sys 0.000s
I20260709 13:06:01.136838 14792 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:01.127187 14825 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:01.153784  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:01.243921  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 220407us
I20260709 13:06:01.248401  9271 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 7.
I20260709 13:06:01.191990 15092 raft_consensus.cc:2804] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:01.245239 15083 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:01.237361 14981 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:01.249974  9192 raft_consensus.cc:2468] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:06:01.304903  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:01.250835 15092 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.307638 15092 raft_consensus.cc:3060] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:01.250860 14792 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.309187  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.310273 14792 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.272424  9198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 1138) took 1239 ms. Trace:
W20260709 13:06:01.290396  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 52237.1us
I20260709 13:06:01.293401  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:01.314822  9198 rpcz_store.cc:276] 0709 13:06:00.032521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.104450 (+ 71929us) service_pool.cc:224] Handling call
0709 13:06:00.104811 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:00.724644 (+619833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:01.272098 (+547454us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b5400434ed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:01.272357 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1330974976}
I20260709 13:06:01.316115  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:06:01.264617 15089 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:06:01.297973  9087 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:06:01.350281  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 52382.3us
W20260709 13:06:01.275539 15019 consensus_meta.cc:287] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.809s	user 0.011s	sys 0.001s
I20260709 13:06:01.351514  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:06:01.303269 15046 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.645s	user 0.003s	sys 0.000s
W20260709 13:06:01.292701 15047 log.cc:927] Time spent T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.799s	user 0.005s	sys 0.000s
I20260709 13:06:01.358515  9087 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:01.245798 14986 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.277004  9195 raft_consensus.cc:606] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:01.350028  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.279801  9197 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
W20260709 13:06:01.357934  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 274437us
I20260709 13:06:01.282935 15073 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:01.379731  9190 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:01.277760  9193 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:06:01.259924 15049 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:01.394850  9198 raft_consensus.cc:1240] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:01.389048  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:01.358584  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 220999us
I20260709 13:06:01.293434 14809 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:01.349073  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:06:01.406587  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 157128us
I20260709 13:06:01.407757  9825 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:01.411175  9266 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:01.412490  9197 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:01.256173 15083 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.407797  9828 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:01.379436 14914 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:01.380338 15010 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:01.270622 14720 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:01.286581 15085 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.383688 14662 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:01.284600 14844 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.293663 15093 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.120s
I20260709 13:06:01.255615 14826 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:01.390574 15086 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:01.414261 14426 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.000s	sys 0.012s
I20260709 13:06:01.430945 14773 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:01.465355  9121 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:01.379132 15019 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.411367 15073 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
W20260709 13:06:01.465673 14426 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.001s	sys 0.013s
W20260709 13:06:01.489297  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 140305us
I20260709 13:06:01.437508 14985 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.463328 14914 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:01.491971  9189 raft_consensus.cc:2393] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 9: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:01.428892 14809 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:01.435806 15010 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:01.496414  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 86485.5us
I20260709 13:06:01.462555 14931 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:06:01.480710 14773 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:01.460283 14662 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.508926  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 3196) took 1578 ms. Trace:
I20260709 13:06:01.496343 14914 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:01.500031 14996 consensus_meta.cc:287] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.710s	user 0.010s	sys 0.002s
I20260709 13:06:01.530414 14844 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:01.529166 14996 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.532019  9114 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:06:01.510231 15094 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:06:01.534885 15001 consensus_meta.cc:287] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 1.065s	user 0.002s	sys 0.010s
I20260709 13:06:01.535380 15001 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.536434 15106 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:01.537724 15095 raft_consensus.cc:2804] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:01.538199 15095 raft_consensus.cc:697] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:01.538615  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:01.538982  9264 raft_consensus.cc:1240] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:01.510425  9263 rpcz_store.cc:276] 0709 13:05:59.930487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.930932 (+   445us) service_pool.cc:224] Handling call
0709 13:05:59.956751 (+ 25819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:59.968371 (+ 11620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:59.968396 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:59.968848 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:06:00.300076 (+331228us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:06:00.300125 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.300200 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:01.302531 (+1002331us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:06:01.503216 (+200685us) raft_consensus.cc:1685] finished
0709 13:06:01.503246 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.508858 (+  5612us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11190,"raft.queue_time_us":60997,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":9106816,"thread_start_us":330798,"threads_started":1,"wal-append.queue_time_us":586439}
W20260709 13:06:01.540418 14884 consensus_meta.cc:287] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.754s	user 0.007s	sys 0.004s
I20260709 13:06:01.540808 14884 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.541785 14662 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.542917 15098 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:01.543478 15098 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.544145  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:01.544615  9267 raft_consensus.cc:2393] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 15: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:01.544708  9120 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:01.544657 14821 consensus_meta.cc:287] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.852s	user 0.008s	sys 0.004s
I20260709 13:06:01.545650 15098 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:01.546236 15098 consensus_queue.cc:1059] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:01.544848 14914 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:01.552328  9194 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:01.553282  9192 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
W20260709 13:06:01.507375  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 79536.2us
I20260709 13:06:01.555189  9862 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:01.555310  9828 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:01.514925 15010 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.517221 15019 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.517443 14826 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:01.517823  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 1131) took 1673 ms. Trace:
I20260709 13:06:01.576229  9196 rpcz_store.cc:276] 0709 13:05:59.835692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:59.880338 (+ 44646us) service_pool.cc:224] Handling call
0709 13:05:59.901587 (+ 21249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:05:59.924487 (+ 22900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:05:59.924518 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:05:59.969568 (+ 45050us) log.cc:844] Serialized 26 byte log entry
0709 13:06:00.283513 (+313945us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:00.297881 (+ 14368us) log.cc:844] Serialized 14 byte log entry
0709 13:06:00.307606 (+  9725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:00.307702 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:00.365703 (+ 58001us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b48000030d8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765b60daa1 00007f765b604b37 00007f765b6049ca 00007f765b6fc26f 00007f765b6fbf37 00007f765b71055c 00007f765b710508 00007f765b710488 00007f765b710443 00007f765b70f6ec 00007f765a639a96
0709 13:06:00.367485 (+  1782us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b180040afe0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535f73f3 00007f76535f97c1 00007f76535f9779 00007f76535f9709 00007f76535f96d1 00007f76535f89cd 000000000038f354 000000000038f199 00007f76535da83a
0709 13:06:01.292101 (+924616us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:06:01.508913 (+216812us) raft_consensus.cc:1685] finished
0709 13:06:01.508941 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:01.509077 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5475,"prepare.queue_time_us":17556,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":19243,"raft.run_cpu_time_us":2770,"raft.run_wall_time_us":84194,"spinlock_wait_cycles":57517312,"thread_start_us":313496,"threads_started":1,"wal-append.queue_time_us":440734}
I20260709 13:06:01.519093 14809 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:01.578699 14981 raft_consensus.cc:3055] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:01.519832 14903 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:01.579243 14981 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:01.520309 14720 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:01.521895 15083 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:01.522773 14986 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:01.497740  9008 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 017b65b967c44e7688e66e7aab235b62 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:06:01.507001 15099 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:01.523720 14773 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:01.580807  9008 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 266826us
I20260709 13:06:01.581418 14773 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:01.524593 15104 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:01.585801 15049 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:01.587437 14981 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.588516 14981 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:01.587324 15049 consensus_queue.cc:1059] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:01.589910 15049 raft_consensus.cc:3055] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:01.590250 15049 raft_consensus.cc:740] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:01.591003 15049 consensus_queue.cc:260] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:01.500252 14815 consensus_queue.cc:1059] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:06:01.539280 15095 consensus_queue.cc:237] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.542938 14884 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.553977 14996 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.572320 15001 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.519307 15085 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.579372 14903 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:01.653909  9118 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:01.595101  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:01.526338 15092 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.654834  9121 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:06:01.656247 15092 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:01.597023  9120 raft_consensus.cc:2468] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
W20260709 13:06:01.600963  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 93950.1us
I20260709 13:06:01.607492  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.657719  9116 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 18: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:01.608914  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.658608  9194 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:01.658972  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:01.660586 15100 consensus_queue.cc:1059] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:01.662392 14821 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:01.662746 14836 raft_consensus.cc:3055] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:01.663115 14821 raft_consensus.cc:993] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:06:01.663182 14836 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:01.610230 14792 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:06:01.615447  9191 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:01.618227  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:01.621829 14914 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.667861  9121 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:06:01.668238 14914 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.635085  9123 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:06:01.637517 14915 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:01.644577  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:06:01.645067  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 89831.5us
W20260709 13:06:01.675083  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.675242  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:01.675573  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.669157  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:01.666704  9264 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:06:01.672394 14915 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:01.681118  9267 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:06:01.681545  9267 pending_rounds.cc:85] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 9
I20260709 13:06:01.681762  9267 pending_rounds.cc:107] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:06:01.681910 14915 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.684450 14426 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:01.681967  9267 raft_consensus.cc:2889] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:01.685048 14996 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:01.669338 15075 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:01.669165 15067 raft_consensus.cc:3055] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:01.688544 15067 raft_consensus.cc:740] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:01.689693 15067 consensus_queue.cc:260] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.690186  9193 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:01.690264  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:01.691524  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.692059  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:01.692781  9195 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:06:01.693749  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.663723 14821 raft_consensus.cc:1081] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:06:01.652796 14815 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:06:01.695914 14815 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:01.666453  9191 raft_consensus.cc:606] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:01.696619  9191 raft_consensus.cc:614] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:01.697048 14815 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.698076 14815 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:01.698491  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:01.695377  9190 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:06:01.678205  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:01.700922  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:01.695844  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.701455  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:01.702567  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:01.704650 15001 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:01.705274 15001 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:01.705945 15001 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:01.678102  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:01.710495 15085 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:01.712616 14720 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:01.713147 14720 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.714174  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.714340 14825 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:06:01.714737 15065 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:01.715327 14825 raft_consensus.cc:697] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:01.716692  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.714807  9190 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:01.717249  9191 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:01.717340  9114 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:01.724989  9235 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:01.725730  9192 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:01.726362  9192 raft_consensus.cc:606] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:01.721362  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:01.716408 14825 consensus_queue.cc:237] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:01.732467  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.733417  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.734220  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:01.732157  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:01.738238  9190 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 18.
I20260709 13:06:01.739044  9857 raft_consensus.cc:2393] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 16: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:01.736558  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.739517  9264 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:06:01.746914  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:01.746202  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:01.747992  9194 raft_consensus.cc:2393] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 17: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:01.748642  9089 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:01.746212  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:01.750025  9189 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:01.738256  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:01.751068  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:01.751940  9862 raft_consensus.cc:2393] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:01.752702  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.760305  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.761448  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.762486  9197 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:01.768733  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.769097  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:01.769536  9117 raft_consensus.cc:2368] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16.
I20260709 13:06:01.771935  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:01.772392  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:01.775066  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:01.715504 14720 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:01.695433 14736 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:01.720563 14984 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:01.778966  9862 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:06:01.692080 15049 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:01.748770 15052 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:01.753046 14915 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:01.816016  9089 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:01.817656  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:01.754336 15010 raft_consensus.cc:2804] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:01.809703  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:01.761190 14809 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:01.812681  9191 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:06:01.751802 15104 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:01.737141 15075 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:01.761708 14662 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:01.777773 15001 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:01.824664  9089 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:01.780103 14986 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:01.753885 14884 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
W20260709 13:06:01.845376  9235 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:01.851706  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:06:01.791635 15074 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.002s	sys 0.012s
I20260709 13:06:01.853641  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:01.856078  9853 raft_consensus.cc:1275] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:01.853185  9164 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:01.859244  9198 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:01.856256  9862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:06:01.870951  9117 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:06:01.872475  9862 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:01.875609  9862 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:01.791544 15064 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:06:01.810287 15061 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:01.828387 14996 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:01.882457  9121 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:06:01.857859 14809 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:01.875257 15065 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:06:01.874357 15074 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.002s	sys 0.013s
I20260709 13:06:01.828157 15073 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:06:01.910104  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:01.828190  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:01.875404 14376 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:01.873055 14985 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:01.870571 15104 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.904094  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:01.906406  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:01.902637 14996 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:01.908052  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:01.923568  9860 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:01.923828  9858 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:06:01.858830 14662 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.944305  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:01.948447  9853 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:06:01.821879 14836 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:01.954802  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 178057us
I20260709 13:06:01.902937 14986 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:01.796393 14903 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.957280  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:01.963156  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:01.921437 15061 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:01.967356  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:01.969229 14809 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:01.971833 15104 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:01.973961  9828 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:01.976434 15117 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:01.976941 15117 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:01.977317 15117 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:01.977442 15061 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.946187 14376 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:01.946996 14985 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.980962 14376 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:01.959920 15075 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.976035 14996 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:01.986362  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:01.959022 14986 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:02.045753  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:02.048053  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.050680  9269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2405) took 1056 ms. Trace:
I20260709 13:06:02.055996  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:02.056082  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.066329  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.077205  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.077594  9267 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:02.078248  9266 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 21 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:02.045010 15010 raft_consensus.cc:697] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:02.042634 14795 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:02.061825 15106 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.077283 15095 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:02.045747 15049 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:02.133106  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:06:02.158720  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.039851 15001 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:06:02.091566  9858 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:06:02.137409 14986 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.137796 15010 consensus_queue.cc:237] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:02.148471 15115 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:06:02.153137  9118 raft_consensus.cc:1217] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Deduplicated request from leader. Original: 16.12->[18.13-20.14]   Dedup: 18.13->[20.14-20.14]
I20260709 13:06:02.065547 14985 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.166328  9858 raft_consensus.cc:1240] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:02.174021  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 296636us
I20260709 13:06:02.062923  9269 rpcz_store.cc:276] 0709 13:06:00.994608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:00.997235 (+  2627us) service_pool.cc:224] Handling call
0709 13:06:01.548796 (+551561us) spinlock_profiling.cc:243] Waited 957 us on lock 0x7b5800114998. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfb71 00007f76535bfa22 00007f765c4d8c09 00007f765b632ee0 00007f765b6fc0ab 00007f765b6fc131 00007f765b6bdfc0 00007f765b6a8929 00007f765b6a83d2 00007f7654c1fbd4
0709 13:06:01.548989 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:01.549203 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:01.549486 (+   283us) log.cc:844] Serialized 14 byte log entry
0709 13:06:02.044962 (+495476us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:02.045021 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:02.045104 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.045187 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:02.050624 (+  5437us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20812,"prepare.queue_time_us":144,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":57073,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":13733,"spinlock_wait_cycles":2200960,"thread_start_us":494983,"threads_started":2,"wal-append.queue_time_us":405509}
I20260709 13:06:02.178995  9118 raft_consensus.cc:1240] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 13:06:02.177333  9164 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.178949  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.190465  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:02.192272  9858 raft_consensus.cc:1240] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:02.196817  9189 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 18.
W20260709 13:06:02.198727  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 71577.7us
I20260709 13:06:02.203904 14986 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:02.205691  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:02.208189 15128 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:02.209082 14662 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.181670 15052 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.209808 15052 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.210731 15082 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:02.211133 15082 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:02.212031 15052 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.211946 15082 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.187579 15065 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:02.220304  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.220486  9860 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:06:02.221385  9861 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:06:02.157302 14795 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.222234  9826 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.223852  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:02.229952  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:02.302189 15103 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:02.302817 15103 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:02.303678 15103 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.305033  9119 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:02.306156 14981 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:02.306710 14981 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.307514 14795 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.308046 14795 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.308481 14795 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.308904 14981 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:02.309722  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 94627.8us
I20260709 13:06:02.215958  9852 tablet_service.cc:1558] Processing DeleteTablet for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e with delete_type TABLET_DATA_TOMBSTONED (TS 92919b70789041c8a0de0b3525154fd1 not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:42584
I20260709 13:06:02.311100  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.215984 15067 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:02.161830 15106 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.321763  9825 leader_election.cc:400] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16.
I20260709 13:06:02.322224  9825 leader_election.cc:403] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:02.322925 15095 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:02.323691 15095 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16.
I20260709 13:06:02.325053  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.325372  9825 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.326061 15095 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:02.326187  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.327241  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.327658  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.328068  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.328126  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.328704  9271 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:06:02.318147 14984 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:06:02.329202 14981 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:02.329370  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.329581 14984 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:02.329960  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.330076 14984 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:02.330587  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.330670  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.331374  9263 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:02.331017 14984 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.331722  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.329679  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.332465 14984 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:02.333170  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.333745  9272 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:06:02.334714 14800 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:06:02.335088 14815 consensus_meta.cc:287] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.637s	user 0.003s	sys 0.005s
I20260709 13:06:02.335206 14800 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:02.335827  9856 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:06:02.336141  9856 pending_rounds.cc:85] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 13
I20260709 13:06:02.336089 14981 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:02.336619  9856 pending_rounds.cc:107] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:06:02.336933  9856 raft_consensus.cc:2889] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:02.337174 14800 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:02.338076 14981 consensus_queue.cc:1059] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:02.338881  9198 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:06:02.192096 14821 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.121s
I20260709 13:06:02.341178  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.222932 15133 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:02.344532  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:02.259171 15085 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:02.347476  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:02.311715 15104 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.356652  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.362488  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.368533  9238 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.342147 14795 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.358693  9862 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.376118  9826 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:02.342723 15128 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:02.348055 15118 raft_consensus.cc:3055] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:02.348250 15127 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:02.353173 15130 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:02.399191  9862 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:02.395771 15052 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:02.393918  9860 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 18.
I20260709 13:06:02.352671 15085 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:02.353559 15133 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:02.353809 15073 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:06:02.354096 15104 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.388837 15118 raft_consensus.cc:740] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:02.403297 15052 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:02.428926  9853 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 18.
I20260709 13:06:02.468390  9264 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:02.475049 15061 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:02.403705 15130 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.486218  9270 raft_consensus.cc:1240] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:02.491560 15124 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 13:06:02.408869 15127 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.498857  9266 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
W20260709 13:06:02.535351  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.489948 15128 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:02.523516 15106 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:02.541487  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 173045us
W20260709 13:06:02.542366  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 166320us
I20260709 13:06:02.504480 15085 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.505928 15118 consensus_queue.cc:260] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.553067  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.555380  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:02.541283 15075 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:02.508025 15133 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:02.558684  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.561508  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.562047  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.558970  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:02.559511  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.562592  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.560267  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.563661  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:02.563575  9272 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:06:02.565152  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:02.565817  9272 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:06:02.560058  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 156279us
I20260709 13:06:02.560922  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.570071  9269 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:02.584384  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.538784 15103 consensus_queue.cc:1059] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
W20260709 13:06:02.583807 15137 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:06:02.589962  9826 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:02.567845 15118 raft_consensus.cc:3060] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:02.590225  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:02.562834 14826 consensus_queue.cc:1059] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:06:02.622535 15130 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.568425 15150 consensus_queue.cc:1059] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:06:02.592274 15106 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:02.634948  9192 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:02.613314  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:02.616762 15127 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.592742  9271 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:02.626734  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:02.627246  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:02.636953  9190 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:06:02.639883  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49735.3us
I20260709 13:06:02.638689  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:02.609094 15142 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:06:02.662328  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72432.4us
I20260709 13:06:02.662945  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:02.663280  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:02.645013  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:02.664693  9191 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 18: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:02.648167 15091 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:02.591886 14996 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:02.595918 15075 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:02.658483 14915 consensus_meta.cc:287] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.809s	user 0.011s	sys 0.000s
I20260709 13:06:02.659920 15156 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:02.660544 15158 raft_consensus.cc:3055] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:02.661373 15152 tablet_replica.cc:333] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:02.661580 15106 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:02.659842  9265 raft_consensus.cc:2410] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 13:06:02.656463 15147 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:06:02.601894 15148 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.674624  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.673856  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:06:02.670917  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 324478us
W20260709 13:06:02.690400  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 350947us
I20260709 13:06:02.694267  9191 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:06:02.698999  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.688060 15067 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:02.703209  9089 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:06:02.705235  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.688961 15156 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:02.689785 15158 raft_consensus.cc:740] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:02.709144  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:02.721603  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.723165  9854 raft_consensus.cc:1240] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:02.723392  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:02.724238  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:02.724606  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:02.724936  9856 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:02.725744  9862 raft_consensus.cc:2410] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:06:02.728147  9826 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:02.727778  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.688493 14996 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:02.711539 15140 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:02.738812  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.717764 15156 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:02.747792  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.748433  9267 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:06:02.744736  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.749758  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.751230  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.752151 15163 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:02.752668 15163 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:02.753054 15163 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:02.754084 15120 log.cc:927] Time spent T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.705s	user 0.000s	sys 0.008s
I20260709 13:06:02.755162 15159 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:02.755954 15131 consensus_queue.cc:1059] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:06:02.757340 15152 raft_consensus.cc:2243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 LEARNER]: Raft consensus shutting down.
I20260709 13:06:02.758134 15152 raft_consensus.cc:2272] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 LEARNER]: Raft consensus is shut down!
W20260709 13:06:02.762120 15149 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.144s	user 0.001s	sys 0.001s
W20260709 13:06:02.764407 15146 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:06:02.766983  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:02.767880 15130 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:02.768911  9116 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:06:02.770983  9195 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:02.772063  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 43981.9us
I20260709 13:06:02.772330 15140 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:02.772854  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:02.773257 15140 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.767071 14996 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:02.782262  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 77088us
I20260709 13:06:02.726500 15164 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:02.734212 15095 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:02.788167  9161 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:02.805163  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:02.805717  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:02.805754  9862 raft_consensus.cc:1275] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:02.806027  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:02.807912  9854 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:06:02.699579 15148 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.805207  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:02.814203  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:02.815173  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:02.783273 15159 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:02.787621 15127 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:02.819257  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:02.826539  9271 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:02.820191  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.827433  9266 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:06:02.829180  9191 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:02.829592  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:02.830899  9189 raft_consensus.cc:2393] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:02.833963  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:02.837707  9853 raft_consensus.cc:2410] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:06:02.842406  9860 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:06:02.812732 15164 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:02.811084 15131 consensus_queue.cc:1243] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 14.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:06:02.807606 15010 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:06:02.852396  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:02.831576 15001 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:02.839296 14792 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:02.859023 15151 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:02.862551 15160 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:06:02.831715 15150 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:06:02.869145 15148 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.874207 15082 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:02.877360 15162 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:06:02.877606  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.885656 15164 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.887307 15152 ts_tablet_manager.cc:1916] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:02.887880 15104 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:02.888618  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:02.889557 15156 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.864322 14792 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:02.898895 14792 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:02.865645 15001 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.906322 15001 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:02.906917 15117 consensus_meta.cc:287] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.929s	user 0.013s	sys 0.000s
I20260709 13:06:02.907356 15117 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.908347 14903 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:02.909478 15117 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:02.916570  9118 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47150 (request call id 1101) took 1048 ms. Trace:
I20260709 13:06:02.916998  9118 rpcz_store.cc:276] 0709 13:06:01.868402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.879994 (+ 11592us) service_pool.cc:224] Handling call
0709 13:06:02.152940 (+272946us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:02.180496 (+ 27556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:02.916340 (+735844us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b5400254250. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:02.916485 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587382272}
I20260709 13:06:02.919116 15152 ts_tablet_manager.cc:1929] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.12
I20260709 13:06:02.882673 15151 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:02.881485 14800 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:02.920375 14800 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:02.920746 14800 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:02.921424 15151 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:02.923195 15148 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.923708 15148 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.926889  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.927444 15148 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.928164  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:02.928638  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 153366us
I20260709 13:06:02.930526  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.938302  9265 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
W20260709 13:06:02.938304  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.935133  9271 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:02.939961  9162 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:02.938571  9267 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:06:02.952461 15148 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:02.952950 15148 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:02.953277 15148 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:02.919644 15152 log.cc:1199] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-3-root/wals/685fa3ec557f4f3db02ec6ccce1d6f3e
W20260709 13:06:02.956789  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:02.893188 15156 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:02.965349 14376 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:02.968456  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 148303us
W20260709 13:06:02.971058 15145 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.383s	user 0.001s	sys 0.002s
I20260709 13:06:02.976884 15158 consensus_queue.cc:260] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.977505  9826 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:02.977876  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:02.978173 15158 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:02.974143  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:02.974315  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:02.975306  9122 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:02.981946  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:02.982657  9123 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:02.985562 14496 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:02.986083 14496 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.988271 14496 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:02.991807  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:02.992689  9267 raft_consensus.cc:2410] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:06:02.997568  9269 raft_consensus.cc:3055] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:02.997952  9269 raft_consensus.cc:740] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:02.998644  9269 consensus_queue.cc:260] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:02.999083  9825 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.000205  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:03.000911 15157 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:03.006107  9123 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:03.007575  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.006960  9826 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:02.966027 14376 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.016772  9189 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:03.018808 14376 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.026379 14985 raft_consensus.cc:3055] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:03.019414 15074 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:03.026942 14985 raft_consensus.cc:740] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:03.027380 15074 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 18 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:03.028330 15074 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.028401 14985 consensus_queue.cc:260] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.029924 14985 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.030725 15148 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.019142  9269 raft_consensus.cc:3060] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.033639 15067 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:03.009799  9116 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:03.035144 15156 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:03.035668 15156 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.036056 15156 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:03.036347 15096 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:03.036739 15096 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:03.036875 14836 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:03.037627 15096 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.039852 15163 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.042573 15163 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:03.045276 14792 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:03.048669  9828 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:06:03.049786  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.049875 14800 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.051797 14794 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:03.052297 14794 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.053296  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:03.053848  9193 raft_consensus.cc:3055] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:03.054109  9193 raft_consensus.cc:740] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:03.054315 14794 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.054955  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.058387  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:03.054801  9193 consensus_queue.cc:260] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.059027  9189 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:06:03.011865  9825 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:03.047391 14792 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:03.060554  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:03.064710  9189 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:03.065837  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:03.066978  9189 raft_consensus.cc:2393] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 17: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:03.061866  9118 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:03.060640  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:06:03.067847  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 60970.6us
I20260709 13:06:03.058887  9122 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
W20260709 13:06:03.063289  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 50021.1us
I20260709 13:06:03.066282  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.068183  9192 raft_consensus.cc:3060] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.060644  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:03.070604  9860 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:03.068305  9269 raft_consensus.cc:1275] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:06:03.072924  9194 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:03.074155  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:03.074827  9194 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 19.
I20260709 13:06:03.060588  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.071082  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:03.082934  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.083577  9122 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:03.084327  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.084849  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.079298  9193 raft_consensus.cc:3060] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:03.086467  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.086391  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:03.088070  9161 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:03.086719  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.089352  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.090454  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:03.091145  9853 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 19.
I20260709 13:06:03.092674  9120 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:06:03.090399  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:03.089952  9119 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:03.097932  9190 raft_consensus.cc:1240] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:03.098695  9161 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:03.099377  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.100862  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.102136  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.103897 15130 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:03.104278 15130 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:03.104625 15130 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:03.105151 15150 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:03.105865 15150 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:03.106515 15150 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:03.107093 15150 consensus_queue.cc:1243] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:06:03.108913 15148 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.109481 15161 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:03.109673 15177 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.103919 15171 raft_consensus.cc:2804] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:03.110244 14903 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:03.110571 15171 raft_consensus.cc:697] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:03.110423 15085 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:03.111979 14376 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:03.112262  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 63672.6us
I20260709 13:06:03.108911  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.117357  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:03.117429  9164 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:03.121250  9192 raft_consensus.cc:2468] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:03.122053 14794 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:03.110265 15177 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.126466 14825 consensus_queue.cc:1048] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:03.127987 14915 raft_consensus.cc:2764] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:06:03.111510 15171 consensus_queue.cc:237] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.129634  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:03.122239 14496 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
W20260709 13:06:03.136809  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 125007us
I20260709 13:06:03.137357 15177 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.137729 15163 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:03.138491 15163 raft_consensus.cc:2804] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:03.138959 15163 raft_consensus.cc:697] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:03.139817 15163 consensus_queue.cc:237] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.141506 14662 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:03.142269  9164 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:03.142669 14662 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.143908 14836 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:03.147455  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:03.125852 14794 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:03.149180  9267 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:03.149411  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:03.150245  9265 raft_consensus.cc:2410] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:06:03.151017  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:03.146634  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:03.153752  9189 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:06:03.155831  9266 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:06:03.093402 14884 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:06:03.155869  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:03.166930  9856 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:06:03.111580 14903 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:03.161383  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:03.172364  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:03.173000  9268 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.173766  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:03.090894 14792 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:03.159643  9087 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.169723  9189 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:06:03.169486  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:03.163596  9265 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:03.179927  9121 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:03.174791  9193 raft_consensus.cc:2468] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:03.177520  9272 raft_consensus.cc:1240] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:03.175985  9853 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:03.196303  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:03.175913  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:03.204288  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:03.204703 14908 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:06:03.206286 14800 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.210254  9197 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:03.153203 14794 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:03.213696  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:03.214836  9195 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:06:03.215088  9857 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:03.220546  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:03.172435 14903 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.164746 15127 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:03.200193 15091 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:03.135519 15156 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.238484  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:03.253734  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 94177.4us
W20260709 13:06:03.240168  9238 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:03.236464  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:03.152842 15157 consensus_queue.cc:1059] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:06:03.241248 14903 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:03.247753 15167 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.110291 15172 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.158s
I20260709 13:06:03.252178  9114 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:03.164772 15158 consensus_queue.cc:1048] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
W20260709 13:06:03.239840  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.255519  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.256899  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.220906 15117 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:03.220248 14809 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:03.283300  9858 raft_consensus.cc:2393] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 19: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:03.252986 15091 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.277000  9161 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:03.290299  9164 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.292651  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.296352 14908 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:03.281352 15095 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:03.297849 15095 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.300071 14915 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:03.300711 15095 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:03.288748 15075 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:03.301803 14984 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.303893 15106 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:03.304111  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:03.304942 14908 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:03.305451 15177 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.307266  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.222975 15067 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:06:03.259027 15161 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:03.309451  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.279279 15117 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.321616  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:03.325022 14903 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.331064 15139 raft_consensus.cc:3055] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:03.319530 15010 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.326525 14984 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.315857 14627 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.333379 14662 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:03.334722 15130 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.337850 14794 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:03.309067 15091 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:03.319658 14792 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:03.278911 15127 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.341035  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:03.343209  9857 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:06:03.269268 15160 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:06:03.348489  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.351356  9271 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:03.376652 14800 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.380141  9825 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.290231 15064 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:06:03.296452 15154 consensus_queue.cc:1048] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.296s
I20260709 13:06:03.349411 15104 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:06:03.400349  9861 raft_consensus.cc:1240] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:03.380892 15117 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:03.378600 14809 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:03.391903 15172 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:03.408360  9263 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:03.390671 15139 raft_consensus.cc:740] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:03.394382 14627 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:03.396703 15075 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.288956 15167 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.416002  9856 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:06:03.417471 14800 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:03.432633  9856 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:03.433776 14800 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:03.433498 15075 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:03.420900  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72481.2us
I20260709 13:06:03.436194  9087 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:03.423156  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:03.437047  9194 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:03.381237 15010 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.400732 14792 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.424546 14915 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:03.456568 15185 log.cc:927] Time spent T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:06:03.460022  9190 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:06:03.473631  9194 raft_consensus.cc:2468] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:03.474192 15130 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.474588 15133 raft_consensus.cc:1064] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: attempting to promote NON_VOTER 92919b70789041c8a0de0b3525154fd1 to VOTER
I20260709 13:06:03.474957  9828 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:03.475792 15103 raft_consensus.cc:2804] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:03.476325 15103 raft_consensus.cc:697] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:03.477160 15103 consensus_queue.cc:237] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.477521 15133 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:03.480276 15184 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:06:03.428736 15172 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.444985 15161 consensus_queue.cc:1059] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:06:03.482815 14794 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:03.445029 14792 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:03.484277 15161 raft_consensus.cc:3055] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:03.484704 15161 raft_consensus.cc:740] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:03.485513 15161 consensus_queue.cc:260] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.486682 15161 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:03.488997 15075 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:03.492312 14984 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:03.445458 15164 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:03.496294  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:03.497378 15167 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.498238  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:03.503829 15190 log.cc:927] Time spent T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:06:03.512738  9268 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:03.512943  9263 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
W20260709 13:06:03.513752  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 72792.5us
I20260709 13:06:03.533388  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:03.541155 15067 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.541155  9826 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:03.476018 14627 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:03.473101 15156 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:03.556015  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 175944us
I20260709 13:06:03.558401 15127 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.559882 14792 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.562414 15117 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.559137 15091 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.517889  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:03.569358  9854 raft_consensus.cc:2368] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13.
I20260709 13:06:03.571434  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:03.578234  9164 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:03.472821 15139 consensus_queue.cc:260] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.584370  9860 pending_rounds.cc:85] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 12
I20260709 13:06:03.594951  9860 pending_rounds.cc:107] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:06:03.595247  9860 raft_consensus.cc:2889] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:03.595499  9238 leader_election.cc:400] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13.
I20260709 13:06:03.579471 14984 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:03.585736 15010 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.540699 15075 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:06:03.593012  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:03.611989  9854 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:03.593873  9164 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:03.612681 15096 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:03.613447 15082 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:03.613956 15082 raft_consensus.cc:697] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:03.614817 15082 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.616196 14792 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.616969 15062 raft_consensus.cc:993] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:03.617496 15062 raft_consensus.cc:1081] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
W20260709 13:06:03.620644  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 102823us
I20260709 13:06:03.628012  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:03.595819  9238 leader_election.cc:403] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:06:03.600191 15139 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:06:03.603978  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.595155 15079 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:06:03.638751  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45345us
I20260709 13:06:03.577169 15067 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.642326  9862 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:06:03.644418  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:06:03.643548  9238 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Couldn't send request to peer 92919b70789041c8a0de0b3525154fd1. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:03.654024  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:03.657404 15167 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:03.662834  9116 raft_consensus.cc:1275] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:03.663816  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.664721  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.659473  9862 pending_rounds.cc:85] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 11
I20260709 13:06:03.670503  9862 pending_rounds.cc:107] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:06:03.675817  9862 raft_consensus.cc:2889] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:03.631999 14984 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:03.683378  9235 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
W20260709 13:06:03.692284  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:03.634779 14809 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:03.687615  9271 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:03.635933 15177 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:03.698433  9263 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:06:03.698865  9263 pending_rounds.cc:85] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 11
I20260709 13:06:03.699086  9263 pending_rounds.cc:107] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:06:03.699308  9263 raft_consensus.cc:2889] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:03.646122 14985 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13.
I20260709 13:06:03.699834  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.705588  9235 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.715847  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.716943  9121 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:06:03.709105 15010 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:03.704387 15160 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:03.717753 15164 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:06:03.682574 15196 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
W20260709 13:06:03.782689  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 57079.2us
I20260709 13:06:03.717811 15049 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:03.706666 14914 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
W20260709 13:06:03.782202 15191 log.cc:927] Time spent T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:06:03.800984 15049 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:03.781741 15178 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:03.783149 15133 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:03.787536 15160 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:03.794898 14985 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:06:03.825572 15160 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:03.925544  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 50608us
W20260709 13:06:03.896008  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 180292us
I20260709 13:06:03.845438 15164 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:03.848347 15080 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:03.875514 15170 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:03.942445 15204 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:06:03.945137  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:03.951979  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:03.952677  9191 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:03.813647 15049 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:03.947979  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.949486  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:03.949712  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:03.962160  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:03.962774  9117 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:03.962975  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:03.963584  9235 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:03.930675 15091 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.963579  9197 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:03.981750  9191 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 20.
I20260709 13:06:03.988377 15061 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:03.989100  9192 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:03.990083  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:03.955775 15117 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:04.013531  9189 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:03.990163  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:04.014715  9192 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 20: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:03.964026 15130 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:03.963936 15067 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:03.846349 14984 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.026146  9198 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:03.982734 15065 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.032924 15065 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:04.033406 15210 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:06:03.951575 15080 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.035059 15065 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.035918 15080 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:03.955616 15170 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.038298 15170 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:04.039870 15130 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:04.040958  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.041651  9196 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:06:04.041133 15130 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:03.943787 14736 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index 8 to 12, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:03.951046 15164 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:04.030978  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.055874 15117 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:04.056736 15117 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.060917  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.062665 15117 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:03.976631 15133 consensus_queue.cc:1048] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:06:04.079202  9117 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
I20260709 13:06:04.084573 14985 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:04.088840  9196 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:06:04.104882 15209 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:06:03.245457  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:03.144409  9008 catalog_manager.cc:5028] TS 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): tablet 685fa3ec557f4f3db02ec6ccce1d6f3e (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
W20260709 13:06:04.117766  9008 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 973270us
W20260709 13:06:04.093039 15203 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 13:06:04.157572  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:06:04.159891  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 141657us
I20260709 13:06:04.149247 15177 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.109959 14985 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.178810  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.179869  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.180532  9123 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:04.180575 15130 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.184051  9123 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 8 to 12, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:04.189333  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:04.181288 15049 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:04.169222 15177 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.200299 14985 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.201244 15172 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:04.211387 15177 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:04.212198 15218 log.cc:927] Time spent T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:06:04.220073  9114 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:04.221844  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.224134 15164 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:04.224838 15096 consensus_meta.cc:287] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.612s	user 0.006s	sys 0.006s
I20260709 13:06:04.226697 14985 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:04.227079 15205 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.511s	user 0.001s	sys 0.000s
I20260709 13:06:04.215296  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.249926  9855 raft_consensus.cc:2410] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:06:04.252460 14809 consensus_meta.cc:287] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.000s
I20260709 13:06:04.253008 14809 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:04.211390 15139 consensus_meta.cc:287] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.000s
W20260709 13:06:04.260950 15202 log.cc:927] Time spent T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.290s	user 0.000s	sys 0.000s
I20260709 13:06:04.265568  9861 raft_consensus.cc:1240] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:04.267563  9858 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1751) took 1119 ms. Trace:
I20260709 13:06:04.268056  9858 rpcz_store.cc:276] 0709 13:06:03.147924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.148957 (+  1033us) service_pool.cc:224] Handling call
0709 13:06:04.265101 (+1116144us) spinlock_profiling.cc:243] Waited 86.1 ms on lock 0x7b540048c954. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:04.267471 (+  2370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198144768}
I20260709 13:06:04.269232 15010 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:04.269783 15010 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:04.241258 15064 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:04.270680 15064 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.271095 15064 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:04.194900 15130 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.283484 15010 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.309839 15154 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:06:04.333704 15207 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 13:06:04.340313 15106 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:04.341343  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1758) took 1037 ms. Trace:
I20260709 13:06:04.344153  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.341796  9857 rpcz_store.cc:276] 0709 13:06:03.303599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.409132 (+105533us) service_pool.cc:224] Handling call
0709 13:06:03.409572 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:03.409921 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:06:03.531894 (+121973us) log.cc:844] Serialized 14 byte log entry
0709 13:06:03.596599 (+ 64705us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.260354 (+663755us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:06:04.337091 (+ 76737us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b4c0017fb00. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f76549b0a20 00007f76549c1a29 00007f7654987644 00007f76549aef2d 00007f76549aeb93 00007f76566df18b 00007f76566dfcb5 00007f76566e1004 00007f76566e0729 00007f76566e5d96 00007f76566daadb
0709 13:06:04.338790 (+  1699us) log.cc:844] Serialized 494 byte log entry
0709 13:06:04.338949 (+   159us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:04.338983 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.339071 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:04.341150 (+  2079us) raft_consensus.cc:1685] finished
0709 13:06:04.341169 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:04.341287 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3912,"prepare.queue_time_us":195282,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":30,"raft.queue_time_us":5580,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":126254720,"thread_start_us":154162,"threads_started":2,"wal-append.queue_time_us":394010}
I20260709 13:06:04.309963 15160 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.346568 15154 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.363265 15130 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.367372  9853 raft_consensus.cc:1240] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 18. Current term is 19. Ops: []
W20260709 13:06:04.370960 15224 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:04.372499 14903 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.376183 15064 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.377182  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40106 (request call id 1671) took 1248 ms. Trace:
I20260709 13:06:04.377593  9855 rpcz_store.cc:276] 0709 13:06:03.129222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.129377 (+   155us) service_pool.cc:224] Handling call
0709 13:06:04.376984 (+1247607us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54004a3dd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:04.377125 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2568132992}
I20260709 13:06:04.378094 15154 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.378866 15160 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.378985 15064 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.393922  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:04.394551  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.394784  9855 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 20: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:04.395277  9190 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 18: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:04.397945  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:04.391498 15208 log.cc:927] Time spent T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.259s	user 0.000s	sys 0.002s
I20260709 13:06:04.404147  9854 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:06:04.405210  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.405906  9858 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:06:04.407238  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:04.407912  9193 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:04.408165 15196 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:04.410624  9195 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:04.412102  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.412750  9197 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:06:04.417737  9193 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:06:04.418318  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.425475  9861 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:06:04.426221  9853 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1753) took 1253 ms. Trace:
I20260709 13:06:04.419297  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:04.427510  9858 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:04.426631  9853 rpcz_store.cc:276] 0709 13:06:03.173320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:03.236932 (+ 63612us) service_pool.cc:224] Handling call
0709 13:06:03.237311 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.369467 (+1132156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.425967 (+ 56500us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540065fed4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:04.426133 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2598417792}
I20260709 13:06:04.412786  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.434003  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:04.440090  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:04.440766  9263 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:04.444864  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.443439  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:04.450603  9269 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:06:04.451205  9269 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:04.451555  9269 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:04.452509  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:04.455891  9195 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:06:04.460813 14376 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:04.464510  9272 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:04.469910  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:04.471097  9115 raft_consensus.cc:2393] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 20: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:04.471712  9122 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:04.473584  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.480183 15010 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.480965 14496 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.382046 14736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:39054 (request call id 2) took 2660 ms. Trace:
I20260709 13:06:04.483110  9858 raft_consensus.cc:2410] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:06:04.484244  9269 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.482177 14376 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:04.485387  9265 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:04.486449  9263 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
I20260709 13:06:04.484668  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.489307  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:04.452049 15198 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.494190  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.493484  9269 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.490350 14376 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.481639 15091 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:04.488682 15201 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.488183 15158 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.481555 14736 rpcz_store.cc:276] 0709 13:06:01.721638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.722236 (+   598us) service_pool.cc:224] Handling call
0709 13:06:02.769373 (+1047137us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b5400436820. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f7656661520 00007f76566eab42 00007f76566e9800 00007f76566e29fa 00007f765b6aa4c9 00007f7654c26c54 00007f7654c26be2 00007f7654c26b11 00007f7654c26a8c 00007f7654c25d42
0709 13:06:02.814199 (+ 44826us) log.cc:844] Serialized 494 byte log entry
0709 13:06:02.901967 (+ 87768us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b540010c054. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566ea735 00007f76566e29fa 00007f765b6aa4c9 00007f7654c26c54 00007f7654c26be2 00007f7654c26b11 00007f7654c26a8c 00007f7654c25d42 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:04.381790 (+1479823us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19232,"raft.queue_time_us":21314,"raft.run_cpu_time_us":1062,"raft.run_wall_time_us":1061,"spinlock_wait_cycles":390626304}
I20260709 13:06:04.488391 14496 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.518170  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.516204  9162 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:04.506662 15100 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:04.504906 15091 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.527961  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:04.524387  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 017b65b967c44e7688e66e7aab235b62 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 3)
I20260709 13:06:04.500531 15198 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.530336 15010 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.521507 14836 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:04.532914 15065 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:04.546818  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.554916  9854 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:04.548815  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.545744  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.560997  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:04.533527 15091 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:04.565536  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.567528  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:04.517771 15158 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.574644  9854 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:04.572851  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:04.583038  9197 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:04.583333  9197 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:04.577477  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:04.583485  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.584021  9861 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:04.584765  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:04.585016  9268 raft_consensus.cc:3055] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:04.585368  9268 raft_consensus.cc:740] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:04.585978  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.586094  9268 consensus_queue.cc:260] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.579994  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.580260  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.537058 15080 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:04.589334  9269 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:06:04.592337  9162 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:06:04.589361  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 46657.5us
I20260709 13:06:04.517613 15201 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:04.595029  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 59340.9us
I20260709 13:06:04.595760  9861 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:04.598413  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.598610  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:06:04.600168 15235 log.cc:927] Time spent T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:06:04.601709  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.631896  9859 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:04.632165  9089 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:04.605772 15198 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.539058 15109 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:06:04.576056 15178 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:04.609007  9238 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:04.609472  9197 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.564154 14836 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.557612 15010 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.612121 15216 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:04.536885 15085 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:06:04.533700 15139 consensus_queue.cc:1059] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:06:04.621626 15158 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:04.626132  9855 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:04.564428 15171 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.587426 15154 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:04.637476  9854 raft_consensus.cc:2468] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
W20260709 13:06:04.655861  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 47012us
I20260709 13:06:04.669075  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:04.673574  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.683626 14914 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:06:04.649466 15065 raft_consensus.cc:2804] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:04.686950 14376 raft_consensus.cc:3055] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:04.688620 14376 raft_consensus.cc:740] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:04.670161 14884 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:04.637693 15100 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:06:04.702816  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:04.681785 15216 consensus_queue.cc:1243] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:06:04.649654 14627 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:06:04.712462  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:04.720156  9115 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:04.668962 15178 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:04.687700 15074 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:04.727172  9269 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:04.723102  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:04.729890  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:04.717810  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.713691 14950 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:04.670079 15171 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.706786 14884 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:04.744383  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:04.746713  9193 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:04.719882 14986 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:04.676771 15154 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.739379 15106 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:04.710003 15222 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:04.702029 15215 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:06:04.747423  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.756788  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:06:04.759949  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 131987us
I20260709 13:06:04.761272 15065 raft_consensus.cc:697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:04.777038  9195 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
W20260709 13:06:04.781786  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44079.5us
I20260709 13:06:04.727198 14984 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.704500 15052 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:06:04.720459 15140 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:06:04.767125  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 109145us
I20260709 13:06:04.751327 14376 consensus_queue.cc:260] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.786618 14376 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:04.747788  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 107897us
I20260709 13:06:04.739050 15073 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:04.727108 15061 consensus_queue.cc:1059] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:06:04.755312 14884 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:04.765056  9115 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
I20260709 13:06:04.783321  9161 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:04.788157 15065 consensus_queue.cc:237] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.789173 15171 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:04.793618  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.799822  9264 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:04.804903 14496 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.805823 15154 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.810365 14836 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.811122 15158 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:04.797200 15080 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:04.811760 15158 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.812021 15080 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:04.814237 15158 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:04.796145 15140 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.817034 15140 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.775621 15103 consensus_queue.cc:1059] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:06:04.818410 15103 raft_consensus.cc:3055] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:04.818818 15103 raft_consensus.cc:740] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:04.819219  9197 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:04.819826 15103 consensus_queue.cc:260] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.786943 15222 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:04.823102 15106 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:04.742707 15096 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:06:04.824168 15096 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:04.824615 15096 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:04.825533 15096 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.826839 15096 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:04.766005 15074 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.828260 15074 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.830756 15074 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.741008 14950 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:04.823670 15222 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:04.831796 15222 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:04.832173 15222 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.832531 15222 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:04.833026 14950 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:04.792670 14984 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.768743 15131 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:04.843101  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:04.839900  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:04.844074  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 46882us
I20260709 13:06:04.840196  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.845093  9087 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:04.845227  9191 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:04.842501  9196 raft_consensus.cc:1275] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:04.847342 15010 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.799289 15255 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:04.849543 15255 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.850354 15131 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:04.814328 15080 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:04.844594  9190 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:06:04.823702 15106 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.837581 14984 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:04.849820  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.869390  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.857697  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.863242  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:04.872010  9854 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
W20260709 13:06:04.861163  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 60769us
I20260709 13:06:04.873034  9238 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:04.863596  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.852670 15178 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:04.861398 15255 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.883399  9854 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:04.882921  9161 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:04.885751  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.888718  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.889447  9854 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 9.
I20260709 13:06:04.890094  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.892737  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.893254  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:04.895195  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.899408  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.900959  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.901885  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.902380  9197 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:04.898352  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:04.903295  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.898438  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.903445  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.868631  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:04.859941 15248 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.906077 15248 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:04.908605 15248 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:04.913712  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 121299us
W20260709 13:06:04.916347  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 71332.9us
I20260709 13:06:04.913000  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
W20260709 13:06:04.918004  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49452us
I20260709 13:06:04.863397 15219 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:04.921032  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:04.853192 15246 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:06:04.922248  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.943881  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.918097  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:04.945302 15158 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:04.945719  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:04.946058  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:04.947875 15109 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.948474 15109 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.945736 15158 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:04.950356 15158 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:04.950979 15109 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:04.952157  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.952426  9122 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:04.953428  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.953533  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.954309  9122 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:06:04.933676  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:04.955899 14903 raft_consensus.cc:2764] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:06:04.957952  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:04.852732 15131 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:04.882035 15215 raft_consensus.cc:2804] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:04.915588 15255 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:04.936542  9264 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 10: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:04.924777  9826 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:04.939184  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.940539  9198 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:04.876607  9014 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:04.865797 15201 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:04.963781  9263 raft_consensus.cc:1275] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:04.935153 14836 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:04.970602  9854 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:06:04.932931 15171 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:04.932909 14984 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:04.972909  9268 raft_consensus.cc:3060] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:04.974180  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:05.000182  9861 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:06:04.974545  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.884354 15074 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:04.984694  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:04.915484 15052 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:04.851684 15091 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.924252 14809 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:04.953437  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.993191  9235 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:04.861287 15106 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:04.972604  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:05.040920  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.960227 15073 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:04.864984 14884 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:04.911752 15178 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:06:05.005637  9269 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:06:05.005848  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:05.054791  9114 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:06:05.008105  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:05.013593  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:04.986521 15085 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:05.035868  9268 raft_consensus.cc:1275] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:06:05.061240  9268 pending_rounds.cc:85] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Aborting all ops after (but not including) 12
I20260709 13:06:05.061537  9268 pending_rounds.cc:107] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:06:05.061812  9268 raft_consensus.cc:2889] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:06:05.004029 15225 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:06:05.028538  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.040587  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:04.993234 15198 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:05.042097  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 49002.4us
W20260709 13:06:05.066622  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 119730us
I20260709 13:06:05.015766 15171 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 20 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:06:05.041669  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 78795.9us
I20260709 13:06:05.045156  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:04.997484 15255 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:04.971537 15249 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:06:05.004387 15215 raft_consensus.cc:697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:06:05.072414  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 147750us
I20260709 13:06:05.067510  9235 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:05.072613 14792 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:05.020515 15052 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.039264 15074 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:05.005709 15133 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:06:05.044687 15219 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:06:05.046419 15073 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.027863 15201 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:06:05.004998 14984 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.084287 14376 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:05.088023  9235 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:05.065891 15242 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:05.041848 15252 consensus_queue.cc:1059] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:06:05.092427  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:06:05.139793  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 70165.4us
I20260709 13:06:05.075174 15104 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:06:05.141795  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 73620us
I20260709 13:06:05.081748 15052 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:05.147210  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 54900.5us
I20260709 13:06:05.089449 15001 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:05.147164  9825 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:05.071383 15085 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:05.092952 15073 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:05.144701  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:05.077095 15198 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:05.174661  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:05.222940  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.125738 15242 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.083432 14792 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.146492 15104 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.148430 15109 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:05.149288 15219 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:05.153132 15001 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:05.228621 15246 log.cc:927] Time spent T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:06:05.282944 15222 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.228224 15242 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:05.285461 15222 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:05.286998 15201 consensus_queue.cc:1048] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:05.243243 15104 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:05.280220 15262 ts_tablet_manager.cc:933] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.278373 15001 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:05.300334 15260 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:06:05.301391 15085 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.302062 15091 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:05.304100 15010 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:05.304808 15010 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:05.307231 15010 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:05.309090 14984 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.235530 15255 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.313181  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:05.231524 15171 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:05.316630 15257 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:06:05.235150 15215 consensus_queue.cc:237] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.259625 15074 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:05.327262 15259 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 13:06:05.346095  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 201441us
I20260709 13:06:05.353588  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:05.379262  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:05.384217 15267 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:05.384768 15267 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.335124 15266 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.385902 15266 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.386458 15133 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:05.429930  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 282814us
I20260709 13:06:05.290167 15271 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 13:06:05.381417 15201 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:05.380755 14800 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:05.467902 15258 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.393s	user 0.001s	sys 0.000s
W20260709 13:06:05.557700  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 469698us
I20260709 13:06:05.469760 15065 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:05.382648 15010 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:06:05.613214 15267 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.334610 14984 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:05.416348 15154 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:05.617627 15052 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.620219 15052 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:05.588899 15001 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.623157 15001 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:05.626081  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.599082 15073 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.574981 15064 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:05.580060  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:05.474943 14884 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:05.578855 15256 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:05.601460 15278 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:05.594066 14800 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.603894 15270 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:05.604429 15275 log.cc:927] Time spent T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 13:06:05.604763 15265 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.312s	user 0.000s	sys 0.000s
I20260709 13:06:05.587323 15201 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.652107 15277 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:05.605871 15272 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 13:06:05.606196 15273 consensus_queue.cc:1048] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:05.604812 15266 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.610479 15242 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.611327 15065 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:05.624933 15158 consensus_meta.cc:287] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.005s
W20260709 13:06:05.585196 15096 consensus_meta.cc:287] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.758s	user 0.021s	sys 0.000s
I20260709 13:06:05.624917 15106 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:05.650267  9862 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:06:05.652130 15201 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:05.627550 15109 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:05.653909  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 73984.5us
I20260709 13:06:05.658607 14792 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:05.651384 15073 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:05.671047  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.685012 15267 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:05.685936 15267 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:05.682866  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.695106  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:05.644308 15064 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.696010  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:05.697135  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:05.674491 15106 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.647087 15278 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.703420  9853 raft_consensus.cc:2955] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 8 to 12, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:05.666205 15266 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:05.646736 15256 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.708921  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:05.649226 15270 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.710814  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:05.714277  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:05.714843  9191 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:06:05.718218 14800 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:05.695433 15109 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.696453 15267 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.721706  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:05.724228  9193 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:05.725427  9860 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:06:05.670430 15158 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.731712  9828 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:05.732702  9164 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:05.733107  9828 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:05.733572 14800 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:05.733979  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:05.734678 14800 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:05.734978 15282 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:05.735592 15282 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.736063 15282 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:05.740425  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:05.741080 15104 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.743234  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:05.743830  9195 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
W20260709 13:06:05.744377  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:05.744936 15256 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.745833 15262 tablet_copy_client.cc:323] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Beginning tablet copy session from remote peer at address 127.7.213.67:38807
I20260709 13:06:05.746266  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:05.746654 15104 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.747264  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:05.749221  9858 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:06:05.734279 15171 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:05.752281  9854 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 18: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:06:05.761754 15286 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:06:05.761664 15270 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.763172  9828 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:05.764225 15270 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:05.764729 15270 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.764803 15278 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.765305 15270 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:05.752501 15171 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:05.768564 15171 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:05.769357 15065 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:05.769426 15106 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:05.711488 15266 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.772627  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:05.773474  9190 raft_consensus.cc:3060] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:05.784555  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:05.786875  9190 raft_consensus.cc:2468] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:06:05.790421  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:05.795535 15266 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:05.786981 15109 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:05.792177  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:05.793565 15064 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:05.792127 15242 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.509804 15242 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.510314 15242 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:05.801115 15270 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.832054 15282 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.514587 15270 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.515034 15282 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.092514  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:06.253072 15178 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:06.515946 15282 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.516171 15178 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.516467 15282 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.517345 15096 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.517853 15096 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.518298 15178 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.518841 15178 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.518958 15282 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.270819  9235 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:06.519498 15282 consensus_queue.cc:1048] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:06.519723 15096 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:06.520277  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 249542us
W20260709 13:06:06.520999 15171 consensus_meta.cc:287] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.752s	user 0.011s	sys 0.000s
I20260709 13:06:06.280561 15298 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.521413 15171 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.522245 15298 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:06:06.524070 15171 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:06.524602 15298 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:06.285935  9193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.017s	sys 0.002s
W20260709 13:06:06.526643  9193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.018s	sys 0.003s
I20260709 13:06:06.523443 14985 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:06.287066 15300 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.527382 14985 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.528023 15300 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.529446 14985 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.529953 14985 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:06.530182 15300 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.530369 14985 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.287988  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.521601 15242 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.533382  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.311913  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:06.332464 15085 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.534715  9115 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:06.534607 15085 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.536616 15085 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:06.342059  9825 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.537647 15171 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.538111 15171 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.538514 15302 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:06:06.539471  9825 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.7.213.65:39939 timed out after 1.571s (SENT) [suppressed 13 similar messages]
I20260709 13:06:06.540200 15171 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.540871 15171 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.541296 14986 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:06.541870 14986 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:06.541386 15171 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.542789 14986 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.544809 15171 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.393909 15103 raft_consensus.cc:3060] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:06.545773 14985 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.546331 14985 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.546727 14985 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.406251 15091 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.548152 15091 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.550258 15091 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.411705 15252 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.422215 14914 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.423403 14376 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.552102 14914 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.552186 14376 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.554740 14376 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.554823 14914 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.426611 15130 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.428779  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:06.555893  9122 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:06.555797 15130 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.558535 15130 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.559160 15130 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.433790  9120 raft_consensus.cc:3055] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:06.436580  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1688) took 1467 ms. Trace:
I20260709 13:06:06.559913  9120 raft_consensus.cc:740] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:06.559973  9862 rpcz_store.cc:276] 0709 13:06:04.968808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.968954 (+   146us) service_pool.cc:224] Handling call
0709 13:06:04.969288 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.669788 (+700500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.436301 (+766513us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54006013d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:06.436519 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565408000}
W20260709 13:06:06.441630 15247 log.cc:927] Time spent T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 1.621s	user 0.006s	sys 0.003s
I20260709 13:06:06.551046 15252 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:05.822875 15109 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:06:06.450939  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.562232  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1272) took 1980 ms. Trace:
I20260709 13:06:06.563058 15252 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.562597  9265 rpcz_store.cc:276] 0709 13:06:04.581905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.582056 (+   151us) service_pool.cc:224] Handling call
0709 13:06:04.582405 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.590573 (+  8168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.590602 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.591008 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.742510 (+151502us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:04.742549 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:04.742645 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.441295 (+1698650us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b24001de4d8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765347d9c4 00007f765347bacb 00007f7653636dc9 00007f7653636a55 00007f765363609d 00007f765626810e 00007f765627f88c 00007f765627ebb6 00007f765627e7d8 00007f7656288221
0709 13:06:06.561950 (+120655us) raft_consensus.cc:1685] finished
0709 13:06:06.561973 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.562169 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13474,"raft.queue_time_us":8490,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":34920,"spinlock_wait_cycles":48082304,"thread_start_us":124022,"threads_started":1,"wal-append.queue_time_us":201771}
I20260709 13:06:06.564664 15154 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:06.562240 15109 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.565670  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:06.566252  9861 raft_consensus.cc:3060] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:06.566962  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:06.567538  9857 raft_consensus.cc:3060] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:06:06.567422  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:06.568377  9853 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 18: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:06.565223 15154 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.480975 14809 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.491271 15104 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:06.567360 15109 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.515163 15270 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.251008 15001 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:06.273546 15296 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:06.516249  9116 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 18: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:06:06.485795 15201 consensus_meta.cc:287] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Time spent flushing consensus metadata: real 0.789s	user 0.012s	sys 0.000s
I20260709 13:06:06.583859  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.584391  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.519306 15178 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.362602 15158 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:06.426347  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.585041 15001 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.560527 15010 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.586382 15178 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.586766 15010 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.581461 15267 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.570938 15154 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.587548 14986 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:06.571342  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 270975us
I20260709 13:06:06.587857 15154 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:06.588406 15154 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.588665  9164 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:06.559672 15130 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.589217 15201 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.539054 15302 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.578310  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.590970 15130 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.578531  9270 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:06:06.575242 14496 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.587675 15267 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.593402 14496 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.571241 14809 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.594967 15267 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.569734 15080 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.588780 15010 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:06.560714  9120 consensus_queue.cc:260] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.612154  9120 raft_consensus.cc:3060] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:06.590317  9859 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:06.606873  9857 raft_consensus.cc:2468] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:06:06.571481  9121 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:06:06.570076 15254 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.583874 15296 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.592398 15302 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.575950  9114 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:06:06.592535 14800 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:06.597054  9122 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:06.620137 14376 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.620626 14376 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.597141  9861 raft_consensus.cc:2468] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:06.571483 15104 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.587061 15001 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.586947 15178 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:06.600494  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 61038.3us
W20260709 13:06:06.608404  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 114013us
I20260709 13:06:06.591461 15130 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.633848 15130 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.634294 15221 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.634912 15221 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.609597 15080 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.636446  9162 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:06.611658 15270 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.590338 15154 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.610052 14496 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.635953 15130 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:06.627629 14800 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:06.648252 14985 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:06.609325 14809 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.641921  9828 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:06.665526 14376 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:06.667507  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 93116.3us
I20260709 13:06:06.617631 15010 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.619874 15252 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:06.672314  9235 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:06.673584 15303 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.671584  9120 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:06.674146 15303 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.665759 15154 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:06.675587 15130 raft_consensus.cc:2804] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:06.678126  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.677961 15154 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.680481 15221 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.681102 15221 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:06.682067 15154 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:06.677985  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.679423  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.657413 15001 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.663949 14800 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:06.685189 15001 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.687274 15001 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:06.665081 15178 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.688408  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.688957 15104 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.688748 15178 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.689065  9268 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:06:06.690176 15201 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:06.690764 15178 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:06.692430 15296 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.680080  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.693624  9116 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:06.696887  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.698457  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.699959  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.700588  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.701781  9120 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:06:06.703061  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:06.704306  9272 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:06.694336 15267 raft_consensus.cc:2804] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:06.705142  9235 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:06.685214 15080 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.620524 15254 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.696147 15130 raft_consensus.cc:697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:06:06.708834  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 62908us
I20260709 13:06:06.711984  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.712100  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.713413  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.718786  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.719285  9192 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:06.714567  9190 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:06.720399  9120 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:06.714803  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:06.721550  9198 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 18: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:06.702306 15201 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.721654  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.722203  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.722985  9264 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:06:06.717981  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.725049  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.725699  9264 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 9.
I20260709 13:06:06.693364 15252 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.693074 15010 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.731711  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.732349  9267 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:06:06.728127  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.710897 15304 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:06:06.733376  9272 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:06:06.734396 15303 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.735234  9281 tablet_copy_service.cc:140] P 1efabcb1befc400aadf101e8276fc1aa: Received BeginTabletCopySession request for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e from peer 92919b70789041c8a0de0b3525154fd1 ({username='slave'} at 127.0.0.1:38650)
I20260709 13:06:06.735781  9281 tablet_copy_service.cc:161] P 1efabcb1befc400aadf101e8276fc1aa: Beginning new tablet copy session on tablet 685fa3ec557f4f3db02ec6ccce1d6f3e from peer 92919b70789041c8a0de0b3525154fd1 at {username='slave'} at 127.0.0.1:38650: session id = 92919b70789041c8a0de0b3525154fd1-685fa3ec557f4f3db02ec6ccce1d6f3e
I20260709 13:06:06.736032  9264 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 10.
I20260709 13:06:06.736937  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.728721  9162 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:06.737412  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.716652 15001 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:06.730156  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.716583 15267 raft_consensus.cc:697] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:06.740711  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.741015  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:06.741806  9267 raft_consensus.cc:2410] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:06:06.738360  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:06.715711 15103 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:06.739832  9266 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:06:06.737975  9198 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:06.746013  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:06.746397  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:06.746977  9194 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:06.717846 15080 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:06:06.751067  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.754449  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.755071  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.755554  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.755585  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.755936  9269 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:06:06.756381  9267 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:06:06.757380  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.757239  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.758702  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.753417  9266 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:06:06.760602  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.745836  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:06.762040  9198 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 18: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:06.752123 15310 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:06.740984 15307 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:06.764338  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.765358  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:06.765832  9267 raft_consensus.cc:3060] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:06:06.770012  9194 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:06:06.756381  9264 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:06:06.741408 15001 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.741902 15158 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.772312  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:06.772418  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:06.773726  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.773131  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.778132  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:06.778700  9194 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:06.780678  9266 raft_consensus.cc:2393] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 18: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:06.776631  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:06.782099  9190 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:06.730261 15201 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.784667  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.754347 15308 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:06.786656  9194 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:06.787973  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.788833  9266 raft_consensus.cc:2468] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 10.
I20260709 13:06:06.787635  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:06.794328  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.802429  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:06.811152  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.811446  9190 raft_consensus.cc:2468] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:06.812688  9826 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:06.777149 15001 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:06.801128  9828 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:06.815470  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:06.817139  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:06.817444  9238 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:06.752307 15103 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.824077  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:06.818385  9198 raft_consensus.cc:2368] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 17. Current term is 18.
I20260709 13:06:06.829515 15254 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.831369 15010 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.834424 14800 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.837807 15103 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.839190  9859 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1776) took 1144 ms. Trace:
W20260709 13:06:06.839888  9144 debug-util.cc:398] Leaking SignalData structure 0x7b08003240a0 after lost signal to thread 8024
W20260709 13:06:06.840458  9144 debug-util.cc:398] Leaking SignalData structure 0x7b080035fca0 after lost signal to thread 9014
I20260709 13:06:06.835628  9193 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:06:06.801353 15310 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:06.851392  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.785594 15158 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.849109 15109 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.856536 15252 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.857077  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.878268  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.878989  9192 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:06.857728 15201 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.879698 14496 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:06.880117 15201 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:06.880548 15201 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.880277 14496 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.880946 15201 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:06.882584 14496 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.858142 15171 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:06.882786  9267 raft_consensus.cc:2468] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:06:06.883536 15171 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:06.883971 15171 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:06.884383 15171 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:06.858552 15154 consensus_queue.cc:1059] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:06.886078 15154 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:06.886478 15154 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:06.822883 15304 raft_consensus.cc:2804] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:06.888159 15304 raft_consensus.cc:697] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:06.889089 15304 consensus_queue.cc:237] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.890036 15282 raft_consensus.cc:2804] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:06:06.890264 15261 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 1.767s	user 0.003s	sys 0.003s
I20260709 13:06:06.890630 15282 raft_consensus.cc:697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:06.780792 15130 consensus_queue.cc:237] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.852993 15310 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:06.852831 15074 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:06.891593 15310 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:06.823644 15267 consensus_queue.cc:237] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.839586  9859 rpcz_store.cc:276] 0709 13:06:05.694765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.694878 (+   113us) service_pool.cc:224] Handling call
0709 13:06:06.839012 (+1144134us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b5400511654. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:06.839137 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2056987904}
I20260709 13:06:06.895011 15270 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.895519  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:06.867470  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.897303  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:06.872048 15109 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.891677 15074 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:06.879710  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.897472  9117 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 20.
I20260709 13:06:06.906160  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1278) took 2313 ms. Trace:
I20260709 13:06:06.874732 15158 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:06.868836 14800 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.907689  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:06.906555  9271 rpcz_store.cc:276] 0709 13:06:04.583793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.584155 (+   362us) service_pool.cc:224] Handling call
0709 13:06:04.584453 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.682482 (+ 98029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.682515 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.824106 (+141591us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b1800460480. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535c52a6 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a
0709 13:06:04.825586 (+  1480us) spinlock_profiling.cc:243] Waited 8.96 ms on lock 0x7b2c00112948. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654fed372 00007f765671fa01 00007f76566e443a 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2
0709 13:06:04.826072 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:06:04.942110 (+116038us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:04.942141 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.896400 (+1954259us) spinlock_profiling.cc:243] Waited 97.6 ms on lock 0x7b54000ebfd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:06.896452 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.896494 (+    42us) raft_consensus.cc:1685] finished
0709 13:06:06.896506 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.896671 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2072,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":2111,"spinlock_wait_cycles":275603328,"thread_start_us":115708,"threads_started":1,"wal-append.queue_time_us":211248,"wal-append.run_cpu_time_us":7461,"wal-append.run_wall_time_us":1304248}
I20260709 13:06:06.912119  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:06.912894  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.908988 15074 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:06.914467  9859 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:06.908265 14496 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:06.919345  9281 tablet_copy_source_session.cc:215] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:06:06.922226 15109 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:06.915525  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.926813  9859 raft_consensus.cc:1275] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:06.928700  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:06.929507  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:06.930740  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.887383 15154 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:06.932778 15154 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:06.934520  9859 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:06.916975 14800 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:06.949478  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.895838 15282 consensus_queue.cc:237] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:06.949910  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:06:06.951385  9144 debug-util.cc:398] Leaking SignalData structure 0x7b080034f000 after lost signal to thread 9055
I20260709 13:06:06.952157  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.972532  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:06:06.972744  9144 debug-util.cc:398] Leaking SignalData structure 0x7b0800222fc0 after lost signal to thread 9075
I20260709 13:06:06.974318  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.854782  9238 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:06.953322  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:06.961933  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:06.977291  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 122640us
I20260709 13:06:06.978322  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.987125  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.988116  9116 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:06.988483  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.036172  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.036912  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.037259  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.988744  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:07.038024  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:06:07.038918  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 242116us
I20260709 13:06:07.039089  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.040010  9828 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:06:06.989490  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 236508us
I20260709 13:06:06.990600  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:06.988456  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:06.991629  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56646 (request call id 1192) took 1270 ms. Trace:
I20260709 13:06:06.986909  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:07.047489  9123 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 18: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:07.048595  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.049969  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.050844  9123 raft_consensus.cc:2368] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11.
I20260709 13:06:07.052853  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.053165  9119 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43782 (request call id 3026) took 1424 ms. Trace:
I20260709 13:06:07.053337  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:07.054018  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:07.055115  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.055826  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.056972  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.045176  9191 rpcz_store.cc:276] 0709 13:06:05.721151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.721283 (+   132us) service_pool.cc:224] Handling call
0709 13:06:05.721535 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.721707 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:06:05.754474 (+ 32767us) log.cc:844] Serialized 14 byte log entry
0709 13:06:06.841006 (+1086532us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b1800390d60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:06:06.991188 (+150182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:06.991268 (+    80us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:06:06.991337 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:06.991425 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:06.991563 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6272,"prepare.queue_time_us":31517,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":35,"raft.queue_time_us":588,"spinlock_wait_cycles":8954240,"thread_start_us":149942,"threads_started":1,"wal-append.queue_time_us":1227756}
I20260709 13:06:07.059134  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.062094  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.062500  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.063716  9162 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:07.064064  9115 raft_consensus.cc:2393] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 18: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:07.065330  9162 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:07.053608  9119 rpcz_store.cc:276] 0709 13:06:05.628338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.972051 (+1343713us) service_pool.cc:224] Handling call
0709 13:06:07.052671 (+ 80620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:07.052851 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:07.052877 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:07.052924 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:06:07.052956 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.053015 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.053114 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:06.972738 14800 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.063125  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.072387  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.072723  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:06.985513 15303 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:07.073947  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.074254  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.076092  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:07.051684  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.076404  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:07.090322  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.093684  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.094864  9192 raft_consensus.cc:2410] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:06:07.095643  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.080870  9825 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:07.077169  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 260058us
I20260709 13:06:07.099416  9826 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.100206  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:07.103024  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:07.103474  9195 raft_consensus.cc:2393] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 18: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:07.103703  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.066906 15267 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:07.105329  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.107354  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:07.107156  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.071628 14800 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:07.127556  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:07.126129  9195 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:07.128103  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.074999 14792 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:07.130658  9191 raft_consensus.cc:2393] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:07.087052 15104 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:07.073736 15325 raft_consensus.cc:2764] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:06:07.142997  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.106642 14984 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:07.145391  9195 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:06:07.129038 15302 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:07.097285 15130 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:06:07.080237 15001 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:07.144784 15267 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:07.139389 15074 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:07.181173  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.158095 15325 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:07.164868 15104 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:07.186501  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 82878.3us
I20260709 13:06:07.164851 15302 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:07.193974 15302 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.192705 15074 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:07.198170  9825 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.184162 14985 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:07.200106 14985 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.161823 15270 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:06:07.175520 15323 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:07.206089  9825 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.074545 15308 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.135s
I20260709 13:06:07.184129 15128 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:06:07.218080  9089 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.185765 15324 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:07.222898 15128 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.229162  9162 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:07.229933 14496 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:07.230665 14985 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:07.238440 15332 log.cc:927] Time spent T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:06:07.230706 15128 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:07.239528 15320 log.cc:927] Time spent T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:06:07.242403 15310 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:07.254276 15201 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.255998 14984 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:07.256711  9858 raft_consensus.cc:1240] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:06:07.257853  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.258484  9858 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:07.259387  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.260268 15323 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.261459 15270 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:07.263236 15310 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:07.264344  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:07.266847 15001 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:07.268989  9117 raft_consensus.cc:1240] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:07.214133 15104 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:07.386749  9120 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:07.203042 15171 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.275830 14809 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:07.276947  9271 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:06:07.341352  9825 leader_election.cc:400] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11.
I20260709 13:06:07.389411  9825 leader_election.cc:403] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:07.198941 15325 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.376963 15177 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:06:07.378124  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 399642us
I20260709 13:06:07.381973  9116 raft_consensus.cc:2410] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:06:07.394327  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:07.394402  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 172037us
I20260709 13:06:07.395136  9238 leader_election.cc:400] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 17. Current term is 18.
I20260709 13:06:07.395462  9238 leader_election.cc:403] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:07.274758  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.375777 15252 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:06:07.278049  9859 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:06:07.396885  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:07.397754  9120 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:07.397068  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.400656  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.404568  9859 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:07.412137  9853 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:07.381173 15201 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:07.426013 15001 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 17. Current term is 18.
I20260709 13:06:07.405640 15104 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 10. Current term is 11.
I20260709 13:06:07.429838  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:07.493721  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 115234us
I20260709 13:06:07.522452  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.497639  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:07.510363  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:07.511205  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 383160us
I20260709 13:06:07.523625  9858 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 16: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:07.513808  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.403579  9825 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.514664  9196 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38858 (request call id 2484) took 2596 ms. Trace:
I20260709 13:06:07.526952  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.527776  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.527980 15201 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:07.530114  9858 raft_consensus.cc:2393] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:07.526600  9196 rpcz_store.cc:276] 0709 13:06:04.900919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.901817 (+   898us) service_pool.cc:224] Handling call
0709 13:06:04.902077 (+   260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:04.902288 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:04.902312 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:04.902771 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.014259 (+111488us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:05.014309 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.014394 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:06.889777 (+1875383us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:06:07.457289 (+567512us) raft_consensus.cc:1685] finished
0709 13:06:07.457315 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.497009 (+ 39694us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6101760,"thread_start_us":111117,"threads_started":1,"wal-append.queue_time_us":161392}
I20260709 13:06:07.530736 15201 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:07.531482  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:07.528980  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 125472us
I20260709 13:06:07.532122  9858 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:06:07.532974  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.533875  9858 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:06:07.534574  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.387756 15270 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.536878  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.537590  9859 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:06:07.540611 15254 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:07.544843 15171 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:07.544906  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.546368 15171 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:07.547013 15171 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.549499  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.551134 15171 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:07.552294  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.555233 15082 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:07.558367 15082 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:07.522778 15128 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:06:07.560508 15082 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:07.405627 15177 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.562798  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:07.536453  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.554222 15325 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:07.564476 15177 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:07.553066  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:07.555308  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.565387  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00ff82ce7b7342a4b052db43182f4a99" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:07.565687  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.566421  9194 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:07.535295 14985 consensus_queue.cc:1048] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:06:07.557652 15270 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:07.568166  9858 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:07.497339 15154 consensus_queue.cc:1059] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:06:07.569285  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.569582 15154 raft_consensus.cc:3055] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:07.570068 15154 raft_consensus.cc:740] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:07.570353  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.571456  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:07.554205  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.574999 14800 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.387370 15323 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.578583 14903 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:07.580363  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.581112  9117 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:06:07.580453 15323 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:07.553121  9120 raft_consensus.cc:2468] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
I20260709 13:06:07.577761 14496 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:07.583293  9859 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:06:07.582746  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:07.575485  9858 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 13:06:07.585234  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 50760.1us
I20260709 13:06:07.587210  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.570992 15154 consensus_queue.cc:260] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:07.583252  9855 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.836s	user 0.013s	sys 0.022s
I20260709 13:06:07.583343  9856 raft_consensus.cc:2393] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 11: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:07.589942  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:07.602708  9855 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.856s	user 0.014s	sys 0.022s
I20260709 13:06:07.604984  9238 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.606240 15131 consensus_queue.cc:1059] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:07.609331  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.609597  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:07.604642  9161 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:07.611523  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.610347  9195 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:06:07.614496  9196 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:06:07.614428  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.616353  9858 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:06:07.615890  9161 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:07.617704  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:07.601622 15323 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:07.636847 15323 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:07.637388 15323 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:07.619392 15325 raft_consensus.cc:2804] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:07.641000 15325 raft_consensus.cc:697] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:07.642413 15325 consensus_queue.cc:237] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:07.620452  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.600937 14985 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:07.644935 14985 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:07.645406 14985 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:07.623806 15080 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:07.647950 15080 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:07.648468 15080 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:07.610838 15346 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:07.629101 14800 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:07.631765 15222 consensus_queue.cc:1059] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:07.627213  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.634310  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:07.618846 15001 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:07.658696  9120 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:07.659349  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.660641  9853 raft_consensus.cc:1813] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Handling vote request from an unknown peer 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:07.661285  9853 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:06:07.654505  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:07.659641  9195 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:06:07.614995 15334 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:07.662756  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40106 (request call id 1687) took 2692 ms. Trace:
I20260709 13:06:07.663079  9238 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:07.655992  9236 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.663389  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.665997  9853 raft_consensus.cc:2410] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:06:07.667644  9120 raft_consensus.cc:2468] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 21.
W20260709 13:06:07.668530  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 146157us
I20260709 13:06:07.669099  9196 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:07.655647 15073 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:07.668936  9161 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:07.671662  9853 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:07.664654  9855 rpcz_store.cc:276] 0709 13:06:04.967241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.967435 (+   194us) service_pool.cc:224] Handling call
0709 13:06:04.967829 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:04.968095 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:06:04.988457 (+ 20362us) log.cc:844] Serialized 14 byte log entry
0709 13:06:05.747033 (+758576us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b18002db1a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f76535d9e07 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765627e20b 00007f7656283d23 00007f765628423a 00007f76566eea2a
0709 13:06:07.659399 (+1912366us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:07.659450 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:06:07.659502 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:07.659734 (+   232us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:07.659902 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54903,"prepare.queue_time_us":20197,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":3922,"raft.queue_time_us":57032,"spinlock_wait_cycles":4913536,"thread_start_us":1856126,"threads_started":1,"wal-append.queue_time_us":843792,"wal-append.run_cpu_time_us":1060,"wal-append.run_wall_time_us":1003444}
I20260709 13:06:07.680874  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:07.681929  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:07.656209 15131 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:07.659615 15096 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:07.685724  9120 raft_consensus.cc:1240] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:06:07.686601  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.686838  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.687908  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:07.689049  9144 debug-util.cc:398] Leaking SignalData structure 0x7b08000a78a0 after lost signal to thread 9147
I20260709 13:06:07.713271  9162 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.664275 15334 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:07.716893  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.718905  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:07.720116  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 57112.4us
I20260709 13:06:07.720739  9238 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.715355  9114 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
I20260709 13:06:07.715543  9089 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:07.688552 15250 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:06:07.672674 15219 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:07.724555  9089 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.726132  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.680678 15074 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:07.680404 15128 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:07.690905 15222 raft_consensus.cc:2804] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:07.731822  9238 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.728757 14627 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:07.735625  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.722338 15334 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:07.725332 15001 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:07.749341  9196 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:07.726665 15219 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:07.718043 15096 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.729892 15010 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:07.722491 15349 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:07.730931 15073 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:07.729079 15347 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:07.773183 14809 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:07.767972 15348 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
W20260709 13:06:07.774377  9144 debug-util.cc:398] Leaking SignalData structure 0x7b08001f6620 after lost signal to thread 9222
W20260709 13:06:07.783008  9144 debug-util.cc:398] Leaking SignalData structure 0x7b080018df60 after lost signal to thread 9223
I20260709 13:06:07.775923  9089 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.766273 15010 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:07.749589 15001 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:06:07.797040  9144 debug-util.cc:398] Leaking SignalData structure 0x7b080038cf60 after lost signal to thread 9296
W20260709 13:06:07.797399  9144 debug-util.cc:398] Leaking SignalData structure 0x7b08003bb960 after lost signal to thread 9297
I20260709 13:06:07.750890 15222 raft_consensus.cc:697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 21 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:07.756160 15219 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:07.734385 15074 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.756045 15350 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:07.774905 15347 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:07.738306 15128 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:07.800447 15001 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:07.796003 14903 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:07.799629 15010 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:07.780232 15073 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:07.806654 15347 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:07.835690  9272 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39754 (request call id 2456) took 1120 ms. Trace:
I20260709 13:06:07.836280  9272 rpcz_store.cc:276] 0709 13:06:06.714894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.717741 (+  2847us) service_pool.cc:224] Handling call
0709 13:06:07.835388 (+1117647us) spinlock_profiling.cc:243] Waited 8.58 ms on lock 0x7b54002a3354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:07.835580 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19736832}
W20260709 13:06:07.849887  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 118178us
I20260709 13:06:07.851675  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.822964 15010 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:07.852475 15216 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:07.873891  9144 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6160 after lost signal to thread 9887
W20260709 13:06:07.852802  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 57778.9us
I20260709 13:06:07.865391  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.853148  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:07.896893  9857 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1787) took 1242 ms. Trace:
I20260709 13:06:07.897346  9857 rpcz_store.cc:276] 0709 13:06:06.654213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.865078 (+1210865us) service_pool.cc:224] Handling call
0709 13:06:07.896826 (+ 31748us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:06:07.897725  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 44661.4us
I20260709 13:06:07.901058  9164 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:06:07.883374 15355 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:06:07.848285 15131 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:07.903528  9857 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:07.883589 15154 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:07.895258 15350 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:07.900309 15061 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:07.914132  9825 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:07.903606 15337 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:07.918851  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:06:07.944845  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:07.920192 15171 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:07.926625 15361 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:07.983886 15310 raft_consensus.cc:3055] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:07.984252 15073 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:07.984403 15352 log.cc:927] Time spent T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:06:07.984634  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:07.989239  9858 raft_consensus.cc:1240] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:06:08.004244  9267 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:06:08.004674  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.004976  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:07.861356 15222 consensus_queue.cc:237] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.006003  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:07.981278 15356 log.cc:927] Time spent T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:06:07.993534 15096 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:07.999095  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:08.000555  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:08.001143  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:08.001425  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:07.917860 15302 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:06:07.995406 15323 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.009853 15109 consensus_queue.cc:1059] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:08.012151  9270 raft_consensus.cc:1240] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:08.022436  9270 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39754 (request call id 2450) took 1326 ms. Trace:
I20260709 13:06:07.921286 15350 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.016224  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.017537  9828 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.018194  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:07.981564 15171 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.021731  9194 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:08.012948 15139 raft_consensus.cc:3055] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:08.000524 15310 raft_consensus.cc:740] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:08.022816  9270 rpcz_store.cc:276] 0709 13:06:06.697396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.697515 (+   119us) service_pool.cc:224] Handling call
0709 13:06:06.704444 (+  6929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.012018 (+1307574us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b54001ef8a0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f765665bdc0 00007f76566e61c0 00007f76566e7231 00007f76566e3d8f 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c
0709 13:06:08.022155 (+ 10137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.022380 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66149632}
I20260709 13:06:08.028110  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.013309 15357 raft_consensus.cc:493] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:08.034124  9828 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:07.903520 15360 consensus_queue.cc:1059] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 13:06:08.033532  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:08.036595  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1775) took 2342 ms. Trace:
I20260709 13:06:08.037062  9828 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.013459 15349 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:06:08.026158 15358 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:06:08.051172  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.052397  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.036993  9860 rpcz_store.cc:276] 0709 13:06:05.694243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.694414 (+   171us) service_pool.cc:224] Handling call
0709 13:06:05.704743 (+ 10329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:05.725800 (+ 21057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.036344 (+2310544us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b54007102d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:08.036537 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43139840}
I20260709 13:06:08.069599  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.070849  9235 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.030758 15171 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.076512  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:08.053186  9198 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:08.081936  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:08.081939  9263 raft_consensus.cc:1275] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:08.081480  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:08.084028  9267 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:06:08.030469 15139 raft_consensus.cc:740] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:08.085593  9235 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:08.036644 15073 raft_consensus.cc:2804] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:08.087383  9272 raft_consensus.cc:1275] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:06:08.031656 15128 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:08.033679 15302 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:08.038789 15270 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:08.094378  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:08.036365 15360 consensus_queue.cc:1243] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:06:08.038938 15158 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:08.026547 14985 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.050729 15346 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:08.098685  9270 raft_consensus.cc:2393] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:08.081411 15349 consensus_queue.cc:1243] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 92919b70789041c8a0de0b3525154fd1 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:06:08.104544 14985 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.097190 15073 raft_consensus.cc:697] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:08.109483 15073 consensus_queue.cc:237] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.087262 15359 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:08.036293 15357 raft_consensus.cc:515] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.113158 15357 leader_election.cc:290] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.084439 15080 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.115432 15080 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.116791  9266 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:08.117581  9196 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:08.091341 15061 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:08.094091 15304 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:08.139938  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 58114.6us
I20260709 13:06:08.092671 15302 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:08.141299 15270 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.145829  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.146589 15270 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.143965  9828 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.147897 15158 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:08.159919  9828 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.166112 15010 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.140921 15061 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:08.171233 14984 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:08.171924  9828 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:08.172704 15323 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:08.173099 15362 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:06:08.148699 15302 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.143095 15304 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:08.174116 15061 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:08.181255  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.181871  9087 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.180687 15304 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:08.179368 15109 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:08.194283  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:08.183125 15360 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:08.162333 15219 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.141551 15139 consensus_queue.cc:260] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.200141  9860 raft_consensus.cc:2393] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:08.117239 15178 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 6 to 13, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:08.215561 15360 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.201612  9828 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:06:08.224792  9144 debug-util.cc:398] Leaking SignalData structure 0x7b08002db320 after lost signal to thread 15241
I20260709 13:06:08.229337  9856 raft_consensus.cc:1240] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:08.242937  9828 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.233405  9859 raft_consensus.cc:1275] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:06:08.233536 15096 raft_consensus.cc:3055] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:06:08.236340 15268 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 2.953s	user 0.009s	sys 0.011s
I20260709 13:06:08.244778  9859 pending_rounds.cc:85] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 10
I20260709 13:06:08.252506  9859 pending_rounds.cc:107] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:06:08.252884  9859 raft_consensus.cc:2889] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:08.211333 15350 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.240753 15367 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:08.214977 15198 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.200851 15374 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:08.251762 15096 raft_consensus.cc:740] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:08.256076 15346 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:08.201390 15323 consensus_queue.cc:1059] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:06:08.260175 15368 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:08.216281 15001 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.216528 15109 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.267486 15139 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:08.282318  9197 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2539) took 3275 ms. Trace:
I20260709 13:06:08.198740 15347 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.261508 14984 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:06:08.288867  9197 rpcz_store.cc:276] 0709 13:06:05.006971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.007111 (+   140us) service_pool.cc:224] Handling call
0709 13:06:05.007446 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:05.015250 (+  7804us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:05.015282 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:05.015712 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 13:06:05.119034 (+103322us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:05.119075 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:05.119145 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:08.235755 (+3116610us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:06:08.280380 (+ 44625us) raft_consensus.cc:1685] finished
0709 13:06:08.280410 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:08.282256 (+  1846us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11964,"raft.queue_time_us":8357,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":17628,"spinlock_wait_cycles":15159680,"thread_start_us":103011,"threads_started":1,"wal-append.queue_time_us":267132}
I20260709 13:06:08.271235 15374 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:08.303421  9271 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39752 (request call id 3305) took 1228 ms. Trace:
I20260709 13:06:08.304661  9271 rpcz_store.cc:276] 0709 13:06:07.075190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.075837 (+   647us) service_pool.cc:224] Handling call
0709 13:06:08.303260 (+1227423us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54004452d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:08.303367 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":446615424}
I20260709 13:06:08.305998  9859 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:08.308760  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.304235  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.321079 15262 ts_tablet_manager.cc:1916] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260709 13:06:08.321971 15252 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:08.321774 15302 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.322424 15252 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:08.323911 15255 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:08.324296 15255 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:08.324618 15255 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:08.328202  9854 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1781) took 2619 ms (client timeout 2945 ms). Trace:
I20260709 13:06:08.328891  9862 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1387) took 1764 ms. Trace:
I20260709 13:06:08.301590 15374 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:08.329241  9862 rpcz_store.cc:276] 0709 13:06:06.564576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.565689 (+  1113us) service_pool.cc:224] Handling call
0709 13:06:06.565956 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.652109 (+ 86153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.652164 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:06.652219 (+    55us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:06:06.652264 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.328682 (+1676418us) spinlock_profiling.cc:243] Waited 85.9 ms on lock 0x7b5400700154. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:08.328725 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:08.328839 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197560064}
I20260709 13:06:08.299562 15323 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:06:08.331794 15219 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:08.333185 15334 consensus_meta.cc:287] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
I20260709 13:06:08.333580 15334 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.335912  9267 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:08.336360  9265 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39752 (request call id 3296) took 1759 ms. Trace:
I20260709 13:06:08.336751  9265 rpcz_store.cc:276] 0709 13:06:06.577031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.577180 (+   149us) service_pool.cc:224] Handling call
0709 13:06:06.577454 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:06.595970 (+ 18516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:06.596000 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:06.596037 (+    37us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:06:06.596065 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.336098 (+1740033us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b54002a3854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:08.336162 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:08.336299 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42145536}
I20260709 13:06:08.280193 15104 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:06:08.327793 15096 consensus_queue.cc:260] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:08.328590  9854 rpcz_store.cc:269] 0709 13:06:05.708525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:05.708678 (+   153us) service_pool.cc:224] Handling call
0709 13:06:08.327981 (+2619303us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b5400657a54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:08.328118 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96485632}
I20260709 13:06:08.300943 15010 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.372753 15001 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.339003 15109 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:08.385241  9164 leader_election.cc:343] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Tablet error from VoteRequest() call to peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Illegal state: cannot vote while shut down
I20260709 13:06:08.386632  9164 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:08.378825 15347 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.357095 15171 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.415665 15171 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.381788 15061 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.418076 15061 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:08.389250  9088 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.327955 15371 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 13:06:08.428922  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.389844 15221 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:08.335640 15334 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.394984  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:08.432444  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:08.433233  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:08.397028  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.378175  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.397797  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a51a0795b52e4843959cd3be3d077713" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:08.323289 15252 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.354019 15360 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.419924  9164 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:08.438191  9854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:08.438530  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:06:08.452310  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 265748us
I20260709 13:06:08.394035 15302 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.455582  9087 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.459604  9087 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:08.438084 15314 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:08.466199  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.468936  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.421234 15158 consensus_queue.cc:1048] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:06:08.457338 15252 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:08.474654  9116 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47150 (request call id 1138) took 1480 ms (client timeout 1746 ms). Trace:
W20260709 13:06:08.474869  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 81213.9us
I20260709 13:06:08.457527 15307 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:08.475607  9087 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.462922 15249 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:08.477157  9116 rpcz_store.cc:269] 0709 13:06:06.989320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.037787 (+ 48467us) service_pool.cc:224] Handling call
0709 13:06:08.464683 (+1426896us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54001e5d54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:08.469900 (+  5217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789076352}
I20260709 13:06:08.467360 15250 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:08.440310 15073 consensus_queue.cc:1048] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:06:08.484576  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.471428 15062 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:08.454718 15304 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.479521 15378 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:08.499730 15378 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:08.500213 15378 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:08.505041 15074 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.494825  9087 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.495159  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.506892  9265 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.509919  9087 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:06:08.513003  9198 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38858 (request call id 2505) took 1679 ms (client timeout 1586 ms). Trace:
W20260709 13:06:08.513557  9198 rpcz_store.cc:269] 0709 13:06:06.826946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:06.850618 (+ 23672us) service_pool.cc:224] Handling call
0709 13:06:08.506575 (+1655957us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54001d4f54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:08.506688 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2761782656}
I20260709 13:06:08.468032 15290 consensus_queue.cc:1059] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:06:08.523259 15378 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.493856 14809 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:08.472441 15337 consensus_queue.cc:1048] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:06:08.533965  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.535849 15109 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:08.528411 15349 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:08.508605 15360 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:08.538831  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
W20260709 13:06:08.535848  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 195804us
I20260709 13:06:08.518316 15250 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:08.526172 15378 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.546207  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:08.541553 15360 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:08.544697 15109 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:08.552242 15109 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:08.552717 15109 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.547699 15375 consensus_queue.cc:1059] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:06:08.556773 15375 raft_consensus.cc:3055] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:08.557253 15375 raft_consensus.cc:740] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:08.559715  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:06:08.560817  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 89888.3us
I20260709 13:06:08.495857 15346 consensus_queue.cc:1059] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:06:08.562039  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:08.537142 14809 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:08.564548  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:08.580904  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:08.550565 15367 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:08.582080  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:08.580304  9123 raft_consensus.cc:1275] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:06:08.583158  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:08.583355  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:08.584043  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:08.584563  9122 raft_consensus.cc:1813] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Handling vote request from an unknown peer 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:08.585104  9122 raft_consensus.cc:2410] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:06:08.585793  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:08.586647  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:08.588028  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:08.588629  9115 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:08.593080  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:08.594236  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:08.551090 15360 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:08.600179 14809 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:08.602171 15304 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.603801  9121 raft_consensus.cc:1240] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:08.545032 15250 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:08.605717 15250 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:08.587523  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:08.615027  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:08.615840  9857 raft_consensus.cc:2393] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 16: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:08.611253  9120 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 6 to 13, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:08.618593  9115 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:06:08.620895  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:08.605091  9121 raft_consensus.cc:1275] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:06:08.621239  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:08.622390  9270 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:08.624714  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:08.624747  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.625694  9272 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:08.637426  9196 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:06:08.640067  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.641677  9235 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.643615  9235 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.644299  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:08.645150  9122 raft_consensus.cc:2368] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:08.653889  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.654290 15248 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:08.654954 15248 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.651381 15222 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:08.656303 15222 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.659289 15222 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.646312 15357 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:08.651628 15307 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:08.660971 15307 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.663354 15307 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.579011 15255 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.666426 15248 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.667687 15255 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.672999  9272 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:06:08.675973  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.677228  9265 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:06:08.677861  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.681581  9272 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.686018  9265 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:08.686520  9235 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.697427  9191 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2552) took 1832 ms. Trace:
I20260709 13:06:08.698067  9191 rpcz_store.cc:276] 0709 13:06:06.865492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:07.141873 (+276381us) service_pool.cc:224] Handling call
0709 13:06:08.507486 (+1365613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:08.507655 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:08.507688 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:06:08.507838 (+   150us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:08.507875 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:08.507936 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:08.697175 (+189239us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54001d4f50. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e38ad 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:08.697351 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3117011840}
I20260709 13:06:08.702723  9235 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:08.704037  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 144743us
I20260709 13:06:08.681972 14986 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:08.713634  9235 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:08.688747 15304 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:08.717280  9235 leader_election.cc:400] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:08.721163  9235 leader_election.cc:403] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:06:08.558240 15375 consensus_queue.cc:260] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.706489 15371 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:08.687511 15367 consensus_queue.cc:1059] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:06:08.682435 15369 consensus_queue.cc:1048] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:06:08.709438 15219 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:08.720094 15130 consensus_queue.cc:1059] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:06:08.722385 15133 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:08.722540 15010 consensus_queue.cc:1048] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:06:08.748513 15375 raft_consensus.cc:3060] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:08.741958 15374 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.750978 15374 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.751679  9862 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:08.752858  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.744112 15360 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.746037 15250 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.757994 15250 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.759243  9116 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:08.759807  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.714844 15290 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:06:08.762157 15360 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.781334 15304 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:08.788748 15130 raft_consensus.cc:3055] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:08.789372 15130 raft_consensus.cc:740] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:08.790678 15130 consensus_queue.cc:260] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.791909 15133 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:08.789108  9272 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
W20260709 13:06:08.780299  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 63163.5us
I20260709 13:06:08.806859  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351454182cf49938eb92870b1fb297e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:08.804224  9162 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:08.811339  9235 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.811100 15290 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:08.826020 15103 raft_consensus.cc:3055] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:08.839620 15103 raft_consensus.cc:740] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:08.839709 15290 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:08.840581 15103 consensus_queue.cc:260] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.818218 15219 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:08.831041  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:08.838270  9265 raft_consensus.cc:1240] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:08.838469 15133 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.798218 15109 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.815773 15130 raft_consensus.cc:3060] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:08.805797 14809 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.866747 15109 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.870695 14809 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.872197  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.874600 15270 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:08.875394  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:08.878103  9264 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:08.880021  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:08.881053  9265 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:06:08.840901 15290 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:08.930219  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.932380  9825 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:08.939345  9862 raft_consensus.cc:1240] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:08.940147 15346 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:08.942552  9268 raft_consensus.cc:1240] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:08.944778  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.946769  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.947569  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.948767 15360 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:08.949385 15360 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.950079  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:08.950650  9858 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:08.951680 15360 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.952651 14809 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:08.953187 14809 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.872253 15310 consensus_queue.cc:260] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.960111  9858 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:06:08.955286 14809 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:08.960641  9123 raft_consensus.cc:2468] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:08.910248 15307 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:08.944365 15273 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:08.970073  9860 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:06:08.973822  9268 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:08.984230  9826 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:08.986094 15104 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:08.986613 15104 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:08.983503 15177 raft_consensus.cc:3055] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:08.988817 15177 raft_consensus.cc:740] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:08.990016 15177 consensus_queue.cc:260] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.994467  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.995780  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:08.966516 15307 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:09.004971 15310 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:09.006311 15307 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:09.007580  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.008198 15178 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:08.991587 15177 raft_consensus.cc:3060] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:09.008726 15178 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.009168 15139 raft_consensus.cc:3055] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:09.009616 15139 raft_consensus.cc:740] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:09.010861 15178 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:09.012096  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:09.025794  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:09.026649  9860 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:08.982872 15360 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:09.030110  9853 raft_consensus.cc:1240] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:09.010459 15139 consensus_queue.cc:260] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:08.987457 15104 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.037700 15139 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.029206 15383 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:06:09.045038 15262 ts_tablet_manager.cc:1929] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 19.12
I20260709 13:06:09.057664 15062 raft_consensus.cc:3055] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:09.058334 15062 raft_consensus.cc:740] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:09.058583 15262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:09.033243 15360 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:09.161428  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.161342  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.162786  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.167042  9087 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:09.174836  9117 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:06:09.175602  9117 pending_rounds.cc:85] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 12
I20260709 13:06:09.176177  9117 pending_rounds.cc:107] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:06:09.176685  9117 raft_consensus.cc:2889] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:09.178768  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f89ac8f7b142239d6103bd6ed96037" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:09.180498  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:09.181891  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:09.182845  9115 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.183089  9119 raft_consensus.cc:1275] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:06:09.185631  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:09.179332  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:09.188831  9089 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:06:09.225165  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.225612  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 49938.2us
I20260709 13:06:09.226842  9089 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
W20260709 13:06:09.204548  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.205523  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 185859us
I20260709 13:06:09.117839 15341 raft_consensus.cc:3055] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:09.243150 15341 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:09.244248 15250 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:09.188699  9087 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:09.228291 14903 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:09.229590 14914 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:09.059496 15062 consensus_queue.cc:260] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.233532 14809 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:09.234431 15080 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:09.235433 15096 raft_consensus.cc:3060] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:09.236063 15104 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:09.238119 15109 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:09.238808 15222 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:09.239665 15064 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:09.252559 15080 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.252898 15064 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.248041 14903 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:09.260730  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.252938 15222 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:09.263470 15104 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.267975 15222 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:09.268672 14903 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:06.240259  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:09.270166 15080 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:09.275780 15360 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:09.276741  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:09.256775 14809 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.280006 15368 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:09.280982 15368 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.281463 15349 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:09.282131 15349 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:09.287164 15377 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:06:09.289076  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 100704us
I20260709 13:06:09.263684 14914 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:09.294662 14914 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.295500 15411 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.300845  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.311362  9263 raft_consensus.cc:2393] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 15: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:09.311005 15411 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.305680 15109 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:09.312688 15109 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:09.314383 15411 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:09.313979 15109 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:09.245239 15341 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:09.317404 15341 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:09.317836  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:09.319193  9826 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.285560 14809 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:09.321537  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:09.324209  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 97703us
I20260709 13:06:09.325040 15103 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:09.326622  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.306600 15064 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:09.330216 15416 log.cc:927] Time spent T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:06:09.332139 15289 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:09.333798 15334 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:09.334286 15334 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:09.338583 15334 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.340087 15334 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:09.274178  9014 catalog_manager.cc:5697] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 10, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:09.307812 15368 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:09.308516  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.288334 15377 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.345072  9266 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:09.345453  9235 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:09.298828 15367 consensus_queue.cc:1059] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:09.346832 15367 raft_consensus.cc:3055] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:09.347251 15367 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:09.348140  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:09.349447 15367 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.351718 15367 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:09.353910  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.356128 15349 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.357542  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:09.358376 15377 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:09.364164  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.364671  9121 raft_consensus.cc:1240] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:09.364921 15010 raft_consensus.cc:2804] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:09.365554 15010 raft_consensus.cc:697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:09.366305  9117 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:09.367072 15010 consensus_queue.cc:237] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.372184 15104 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:09.372597 15310 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:09.373396 15310 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.374565 15062 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:09.378755  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.381094  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:09.382153  9116 raft_consensus.cc:2393] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:09.383796 15310 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:09.385532  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:09.357566 15349 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:09.394979  9855 raft_consensus.cc:2410] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:06:09.398005  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40108 (request call id 1803) took 1010 ms. Trace:
I20260709 13:06:09.398471  9855 rpcz_store.cc:276] 0709 13:06:08.387052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:08.397593 (+ 10541us) service_pool.cc:224] Handling call
0709 13:06:09.397667 (+1000074us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b5400733054. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:09.397851 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2204388608}
I20260709 13:06:09.400249  9235 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:09.400517  9857 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:06:09.400921  9857 pending_rounds.cc:85] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 14
I20260709 13:06:09.401175  9857 pending_rounds.cc:107] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:06:09.401412  9857 raft_consensus.cc:2889] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:09.406296  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:06:09.407596  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.413031  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.414078  9269 raft_consensus.cc:2393] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:09.414328  9263 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:09.414440  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.415501  9268 raft_consensus.cc:2393] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 17: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:09.416905 15061 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:06:09.395030  9089 leader_election.cc:343] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Tablet error from VoteRequest() call to peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523): Illegal state: cannot vote while shut down
I20260709 13:06:09.431630  9115 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:06:09.424993  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:09.414099  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:06:09.466543  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 71570.5us
W20260709 13:06:09.468004  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.473985  9116 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 10. Current term is 11. Ops: []
I20260709 13:06:09.477399  9123 raft_consensus.cc:1275] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:06:09.443759 15375 raft_consensus.cc:2764] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:06:09.481106 15249 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:09.482177  9089 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:09.484200 15310 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.494110  9115 raft_consensus.cc:1240] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:09.501668  9164 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:09.508530  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:09.516917  9121 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.520871 14800 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:09.521570 14800 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.524555 14800 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:09.525766  9263 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:09.526932  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.529395  9164 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:09.530323 15221 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:09.531522 15080 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:09.532388 15221 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:09.532629 15080 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:09.548437  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:09.549314  9190 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 19.
I20260709 13:06:09.552511  9190 raft_consensus.cc:1217] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 13.11->[14.12-16.13]   Dedup: 14.12->[16.13-16.13]
I20260709 13:06:09.553483  9190 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:06:09.556354  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:09.558005  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:09.561131  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:09.570914  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:09.578552  9121 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:06:09.510895 15377 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.582578 15377 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.584743 15377 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:09.581516  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 181434us
I20260709 13:06:09.564531 15289 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:09.576922  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:09.565052 14903 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:09.559906 15310 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:09.590332  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:09.594497  9019 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42488 (request call id 241) took 54742 ms (client timeout 15000 ms). Trace:
W20260709 13:06:09.595089  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 90651.4us
I20260709 13:06:09.595389  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:09.585637 15289 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.431159  9014 catalog_manager.cc:5697] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:09.598536  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:09.598959  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cdcabf0b1ec4b319a537ed3662ce253" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:09.602820  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.603524  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 98499.1us
I20260709 13:06:09.611858  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:09.611949  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:06:09.614991  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.644282  9828 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:09.617380  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:09.625314 15310 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:09.632357  9855 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 19: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:06:09.594987  9019 rpcz_store.cc:269] 0709 13:05:14.851621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:14.862678 (+ 11057us) service_pool.cc:224] Handling call
0709 13:06:09.591333 (+54728655us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:00.861720 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:00.925647 (+ 63927us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:00.936790 (+ 11143us) write_op.cc:435] Acquired schema lock
0709 13:06:00.939179 (+  2389us) tablet.cc:662] Decoding operations
0709 13:06:00.985381 (+ 46202us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:01.024278 (+ 38897us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:01.068400 (+ 44122us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:06:01.155169 (+ 86769us) tablet.cc:701] Row locks acquired
0709 13:06:01.155244 (+    75us) write_op.cc:260] PREPARE: finished
0709 13:06:01.155513 (+   269us) write_op.cc:270] Start()
0709 13:06:01.155766 (+   253us) write_op.cc:276] Timestamp: P: 1783602361155418 usec, L: 0
0709 13:06:01.155828 (+    62us) op_driver.cc:348] REPLICATION: starting
0709 13:06:01.156894 (+  1066us) log.cc:844] Serialized 2116 byte log entry
0709 13:06:01.680537 (+523643us) op_driver.cc:464] REPLICATION: finished
0709 13:06:02.138345 (+457808us) write_op.cc:301] APPLY: starting
0709 13:06:02.138486 (+   141us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:02.142828 (+  4342us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:02.142978 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:02.589721 (+446743us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:02.602167 (+ 12446us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:06:02.602237 (+    70us) write_op.cc:312] APPLY: finished
0709 13:06:02.602578 (+   341us) log.cc:844] Serialized 57 byte log entry
0709 13:06:02.602834 (+   256us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:02.602957 (+   123us) write_op.cc:454] Released schema lock
0709 13:06:02.603028 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:02.603564 (+   536us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:02.603595 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2080315,"reported_tablets":44,"spinlock_wait_cycles":6016,"child_traces":[["op",{"apply.queue_time_us":384537,"apply.run_cpu_time_us":12742,"apply.run_wall_time_us":507336,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25037,"num_ops":5,"prepare.queue_time_us":126656,"prepare.run_cpu_time_us":21762,"prepare.run_wall_time_us":584370,"raft.queue_time_us":133437,"raft.run_cpu_time_us":61065,"raft.run_wall_time_us":247544,"replication_time_us":524556,"thread_start_us":706638,"threads_started":4,"wal-append.queue_time_us":325906,"wal-append.run_cpu_time_us":9244,"wal-append.run_wall_time_us":2118642}]]}
I20260709 13:06:09.668428 15377 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:09.638178 15249 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:09.670224 15249 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:09.672765 15249 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:09.641588  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 55892.9us
W20260709 13:06:09.645256  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 163239us
I20260709 13:06:09.678186  9089 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:09.649395  9194 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:06:09.649739  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 53987.4us
W20260709 13:06:09.651858  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.651762  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:09.682240  9268 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:06:09.652871  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:09.684197  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:09.685221 15178 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:09.687093  9271 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:09.689188 15010 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:09.689677 15010 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:09.690918 15010 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:09.693001 15010 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:09.597503 15262 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:40124 (request call id 1381) took 4701 ms. Trace:
I20260709 13:06:09.695382 15262 rpcz_store.cc:276] 0709 13:06:04.858031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.874912 (+ 16881us) service_pool.cc:224] Handling call
0709 13:06:05.328868 (+453956us) ts_tablet_manager.cc:934] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
0709 13:06:08.320250 (+2991382us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:06:08.824846 (+504596us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:09.037533 (+212687us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:09.058167 (+ 20634us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 13:06:09.494303 (+436136us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:06:09.495072 (+   769us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:09.495698 (+   626us) ts_tablet_manager.cc:989] Replica registered.
0709 13:06:09.511794 (+ 16096us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":19373,"tablet-copy.queue_time_us":247892,"thread_start_us":234012,"threads_started":2}
I20260709 13:06:09.697291 15262 tablet_copy_client.cc:806] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 0 data blocks...
I20260709 13:06:09.659536 15080 consensus_queue.cc:1059] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:09.660599  9192 raft_consensus.cc:2393] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:09.646016 15289 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:09.698798 15080 raft_consensus.cc:3055] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:06:09.699148 15080 raft_consensus.cc:740] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:09.660645 15064 consensus_queue.cc:1059] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:06:09.656141  9193 raft_consensus.cc:1240] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:06:09.664350  9161 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:09.667129  9263 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:09.693399 15104 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:06:09.693987  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45038.9us
I20260709 13:06:09.663471  9014 catalog_manager.cc:5697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 13, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 13 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:09.702595 15062 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:09.724277 15062 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:09.724709 15062 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:09.704185 15080 consensus_queue.cc:260] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:09.713096 15061 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:09.714738 15289 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:09.716771  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 52682.1us
I20260709 13:06:09.717976 15222 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:09.717993 14800 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:09.718261 15316 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:09.697973 15262 tablet_copy_client.cc:670] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Starting download of 1 WAL segments...
I20260709 13:06:09.722136  9190 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 11.
I20260709 13:06:09.709038  9164 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
W20260709 13:06:09.752770  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 43806.9us
I20260709 13:06:09.747309 15341 consensus_queue.cc:1048] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
W20260709 13:06:09.762888  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 84803.6us
I20260709 13:06:09.731668 15222 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:09.762265  9122 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:09.727910 15349 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:09.762135 15221 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:09.856446 15310 consensus_queue.cc:1059] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
W20260709 13:06:09.901058  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 51533.3us
I20260709 13:06:09.902102  9161 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:09.918670  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:09.936204  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:09.917831 14800 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:09.984004  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 82008.8us
I20260709 13:06:10.000976  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:09.917616 15062 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:10.007899  9268 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:09.935683 14986 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:06:10.015038  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.011984 15221 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:10.041023  9190 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
W20260709 13:06:10.046517  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 117445us
W20260709 13:06:10.054885  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 65737.7us
I20260709 13:06:10.058807 15431 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:10.072819 15431 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.075357 15431 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.024796  9014 catalog_manager.cc:5697] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 12, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 12 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:10.082630 15426 log.cc:927] Time spent T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:10.084151 15289 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.084473 15062 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:10.059567  9161 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:10.094559  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:10.117375  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 61455.8us
I20260709 13:06:10.118237  9161 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:10.119794 15434 log.cc:927] Time spent T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:06:10.120127 15080 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:10.082762  9014 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index -1 to 6, term changed from 15 to 19, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) added, no longer has a pending config: VOTER ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65), VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67), NON_VOTER 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 19 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } } }
I20260709 13:06:10.122327 15064 raft_consensus.cc:3055] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:06:10.122732 15064 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:10.122026  9014 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 15 to 17, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 17 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:06:10.132328 15424 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:06:10.132510 15432 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:10.131573  9014 catalog_manager.cc:5697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 16, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 16 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:10.134574  9014 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 14 to 17, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 17 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:10.136950 15222 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:10.138520 15222 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:10.137512  9014 catalog_manager.cc:5697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:10.145041 15430 log.cc:927] Time spent T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.128s	user 0.001s	sys 0.001s
W20260709 13:06:10.143576 15429 log.cc:927] Time spent T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:06:10.148378  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.150902  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.152618  9857 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
W20260709 13:06:10.153544 15425 log.cc:927] Time spent T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:06:10.155105  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.143848  9014 catalog_manager.cc:5697] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 12, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 12 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:10.156796  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:10.162009  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 143389us
I20260709 13:06:10.165448 15010 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:10.166239 15010 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.166585 14800 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:10.167295 14800 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:10.168311  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.168980 15010 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.169523 14800 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:10.189525  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.194460  9857 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 16: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:06:10.195328  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 77168.1us
I20260709 13:06:10.196355  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:10.190594  9858 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:06:10.189527  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.189857  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:10.198180  9272 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:10.199635  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:10.200071 15436 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:10.200245  9860 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:10.172159 15349 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:10.200985 15349 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.203156 15349 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:10.207355  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 153360us
I20260709 13:06:10.209414  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.210167  9825 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:10.211136  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.211771  9118 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:06:10.213759  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.214250 15178 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:10.215631  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.216218  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.215651 15435 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:10.217116  9123 raft_consensus.cc:2410] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:06:10.216318  9118 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:06:10.214313  9114 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:10.222611  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.233233  9116 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:10.186655 15250 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:10.235347  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.234745  9122 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
W20260709 13:06:10.238852  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 72733.4us
W20260709 13:06:10.232214  9263 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.011s	sys 0.028s
W20260709 13:06:10.240185  9263 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.012s	sys 0.028s
I20260709 13:06:10.237799  9858 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 19.
I20260709 13:06:10.194927 15367 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:06:10.228359 15103 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:10.257474  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 47418.1us
W20260709 13:06:10.267168  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 207339us
I20260709 13:06:10.267652  9860 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:10.224709 15104 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:06:10.271063 15010 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.200347 15064 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:10.221550 15178 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.287902 15010 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:10.192494  9014 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 15, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 15 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:10.288757 15270 consensus_queue.cc:1048] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:06:10.294936 15270 consensus_queue.cc:1243] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer ce51b61da22f40d4aa0204a6b1028f28 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:06:10.280762  9238 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Timed out: RequestConsensusVote RPC to 127.7.213.66:34417 timed out after 1.586s (SENT)
I20260709 13:06:10.296100  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:10.261261 15375 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.245863 15249 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:10.289254  9858 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:10.233664 15435 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.264122 15377 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:10.311589 15377 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.314934 15377 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:10.302843  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.304260  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.304680 15290 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:10.323063  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:10.327160  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:10.330346  9272 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:06:10.336793  9825 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:10.299515 15103 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.261968 15250 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.347146  9089 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:10.348465  9089 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:10.356963 15367 consensus_queue.cc:1059] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:10.358070 15349 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:10.359279 15349 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:10.361332 15435 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.364100  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.387344  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.389273  9116 raft_consensus.cc:1275] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:06:10.337023 15270 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:10.317811 15249 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.395355 15446 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.399464 15249 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:10.399928  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 125507us
W20260709 13:06:10.401363  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.364390 15103 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:10.424986  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 90233.7us
I20260709 13:06:10.331679 15375 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:10.428910  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 144345us
I20260709 13:06:10.429247 14986 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:06:10.396212 15270 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.398599 15438 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:10.434469  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.439883  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:10.435719  9162 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:10.443502  9197 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 11: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
W20260709 13:06:10.445263  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 154986us
I20260709 13:06:10.435392 14800 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.446802 14800 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.448450  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.449002 14800 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.449118  9189 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:10.449834  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.450399  9192 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:10.450716  9194 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:10.446250  9265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.452612  9265 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:06:10.453473  9193 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 6 to 13, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:10.456233  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:10.456616  9195 raft_consensus.cc:3060] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:10.460287  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.460973  9198 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:06:10.463884  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.463757  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.464682  9118 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 19.
I20260709 13:06:10.465971  9116 raft_consensus.cc:1275] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:06:10.467271  9114 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:10.468194 15250 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.430656 15104 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:10.480316 15449 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.480944 15449 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.483026 15449 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:10.484189  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:10.484810  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.486200 15178 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.486481  9121 raft_consensus.cc:1240] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 9. Current term is 10. Ops: []
I20260709 13:06:10.487723 15010 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:10.399689 15446 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.490052 15446 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.490551  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23636a3a86df4bdf91dc6a7ffe723527" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.491016 15103 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:10.491801  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.436403 15270 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:10.495986  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.497334  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.497915  9119 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:10.500036  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.501144  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.501811  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.501906  9194 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 18.
I20260709 13:06:10.502575  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.503119  9192 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:10.503473  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.503990  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909051b50b2c464ea2b13a15a243817c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.504096  9264 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:06:10.505506  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.507761  9195 raft_consensus.cc:2468] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:10.508086  9235 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:10.512475  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:10.521436  9119 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:06:10.522574  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 86930.7us
I20260709 13:06:10.522878  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.524147 14800 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:10.524808 14800 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:06:10.526157  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 177747us
I20260709 13:06:10.526823  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:10.526998 15443 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.527550 15249 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:10.527531 15443 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.528167 15249 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.528729 15249 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:10.528996  9161 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:10.524327  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:10.536155  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.525831 14800 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:10.537631  9238 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:06:10.568087  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.540773 15222 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:10.537945 15443 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.592145 15458 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:10.563045 15178 raft_consensus.cc:2804] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:10.556933 15341 consensus_queue.cc:1048] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:06:10.615473  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.437346  9014 catalog_manager.cc:5697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:10.618192  9269 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:06:10.588711 15222 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:10.621603 15444 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:10.621836 15448 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:10.622040 15303 consensus_queue.cc:1048] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:06:10.622576 15357 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:06:10.648556  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 59026.9us
I20260709 13:06:10.593652 15431 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:06:10.652454  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.643925 15222 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:10.620824 15178 raft_consensus.cc:697] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
W20260709 13:06:10.660717  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 213965us
I20260709 13:06:10.636049 15375 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.653966 15357 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.649003 15448 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:10.665431  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:10.670274  9825 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:10.659565 15250 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.684870 15250 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:10.687012 15250 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:10.667704 15444 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.670346 15357 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:10.688494  9266 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:06:10.689157  9238 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:06:10.610559  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 56703us
I20260709 13:06:10.672608 15262 tablet_copy_client.cc:538] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:06:10.663180  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:10.704411  9825 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:10.709944  9014 catalog_manager.cc:5697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:10.711485 15447 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:06:10.719349  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 56262.6us
I20260709 13:06:10.672272 15457 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.672833 15448 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:10.692591 15064 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:10.674741 15103 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:10.725399  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:10.728950  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.731839 15463 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:10.725097 15438 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:10.811595  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.817350 15463 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:10.817839 15463 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:10.821928  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:10.823464 15290 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:10.735880 15457 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.722286 15447 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:06:10.823989  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 99612.1us
I20260709 13:06:10.734970 15270 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:10.830026  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 104731us
W20260709 13:06:10.812114 15440 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.463s	user 0.004s	sys 0.000s
I20260709 13:06:10.841266 15377 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:10.843442 15457 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.837749 15444 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.834570  9825 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:10.849012  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.849691  9193 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 11.
I20260709 13:06:10.815410 15222 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.841452  9263 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38650 (request call id 1339) took 1118 ms. Trace:
I20260709 13:06:10.685806 15249 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.738462  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.829856 15290 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.828603 15447 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:10.831197 15411 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:06:10.870497  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 181444us
I20260709 13:06:10.861030  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:10.739478 15436 consensus_meta.cc:287] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.000s
I20260709 13:06:10.728125 15431 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:10.852243 15103 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:10.885504 15270 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:10.854238 15221 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:06:10.887008  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.888187  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.889279  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.856696 15377 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:10.889607  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.854630  9263 rpcz_store.cc:276] 0709 13:06:09.723217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:09.723348 (+   131us) service_pool.cc:224] Handling call
0709 13:06:09.723671 (+   323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:06:09.724227 (+   556us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:06:09.724262 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:06:09.725406 (+  1144us) log.cc:844] Serialized 52 byte log entry
0709 13:06:10.246815 (+521409us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:06:10.246867 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:10.246942 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:10.811348 (+564406us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:06:10.841113 (+ 29765us) raft_consensus.cc:1685] finished
0709 13:06:10.841134 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:10.841373 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":58182016,"thread_start_us":515037,"threads_started":1,"wal-append.queue_time_us":556292}
I20260709 13:06:10.870160 15290 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:10.891924 15436 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:10.893862  9191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1abbfa6c01a144acb9bc31470b44be7e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.894636  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.899314  9114 raft_consensus.cc:1240] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:10.900028  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.889988 15103 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:10.891222 14809 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:06:10.893151  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 84081.1us
I20260709 13:06:10.863031 15104 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:10.907718  9858 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:06:10.909700  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:10.923776  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.923110  9014 catalog_manager.cc:5697] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 14, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 14 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:10.925074  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:10.925913  9014 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 13, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 13 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:10.926625  9189 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:10.877389 15449 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:06:10.881486 15411 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.865691 15178 consensus_queue.cc:237] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.928323  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:10.948262  9189 raft_consensus.cc:606] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:10.949944  9189 raft_consensus.cc:614] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:10.928925  9193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:10.953207 15464 log.cc:927] Time spent T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:06:10.953568 15444 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:10.954077 15444 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:10.954533 15444 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:10.901832 15377 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:10.955852  9197 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:10.957924 15249 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.944092 15104 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.960768  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:10.960837 15357 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:10.964166  9264 raft_consensus.cc:1240] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:10.928421 15289 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:06:10.966122 15222 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:10.927529 15461 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:10.955962 15377 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:10.969853  9263 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:06:10.967196 15461 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:10.972409  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:10.964047 15433 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:10.977316 15411 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:10.949304  9828 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:10.994589 15104 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:10.996047  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.997109  9263 raft_consensus.cc:2393] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 16: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:10.997458 15461 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.000177  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:10.929673 15448 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.004698 15448 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.009361 15447 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.017922 15447 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.019263 15262 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Bootstrap starting.
I20260709 13:06:10.941428 14809 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.024652  9014 catalog_manager.cc:5697] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 10, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.029122 15377 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.031893 15377 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.035691  9268 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:11.034416  9162 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:11.035178 15439 raft_consensus.cc:3055] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:06:11.038154  9163 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.038576  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:06:11.040164  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45662.8us
I20260709 13:06:11.025693 14809 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:11.041410  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 121272us
I20260709 13:06:11.042166  9828 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:10.968518 15463 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:11.048307  9087 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:11.052736  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.053506  9860 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 16: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:11.060521  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:11.061237  9266 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:06:11.043762 15310 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:11.064783  9089 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.038125 15439 raft_consensus.cc:740] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:11.059624  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:11.067850  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:11.078300  9197 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:11.068698  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:11.067011  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.071137 15458 consensus_queue.cc:1059] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:11.072700 15468 consensus_queue.cc:1059] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:11.075450 15222 consensus_queue.cc:1059] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:11.094928 15462 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:11.095466 15462 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.097404  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.098235  9114 raft_consensus.cc:2368] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 19. Current term is 20.
I20260709 13:06:11.066988 15469 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:11.058001 15303 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.046530 15104 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:11.078953  9119 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:11.079061  9266 raft_consensus.cc:2393] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:11.082470  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:11.112089  9195 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:11.086827 15103 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.088212 15349 raft_consensus.cc:3055] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:06:11.027618 15290 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.090626  9118 raft_consensus.cc:1240] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:11.092474  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:11.097564 15462 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.075800 15310 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.100531  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.027156  9014 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 10, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.101920  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:11.105373  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.099952 15469 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.118546  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:11.134877  9195 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:06:11.116315 15349 raft_consensus.cc:740] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:11.095638  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:11.120074 15444 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:11.139485  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 43937.9us
I20260709 13:06:11.114882 15010 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:11.138929  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 86185.8us
I20260709 13:06:11.139393  9264 raft_consensus.cc:2368] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16.
W20260709 13:06:11.137409  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 59574.8us
I20260709 13:06:11.141309  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.140260  9161 leader_election.cc:400] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 19. Current term is 20.
I20260709 13:06:11.142599  9161 leader_election.cc:403] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:06:11.146349  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:06:11.148900  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 43590.2us
I20260709 13:06:11.149613  9238 leader_election.cc:304] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.149744  9855 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:11.150605  9198 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 17: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:11.152077  9195 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:11.152580  9195 raft_consensus.cc:606] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:11.146993  9120 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:11.154060  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:11.154599 14986 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.132969 15249 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:11.133296 15310 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:11.159835 15103 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.137250 15357 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.116411 15303 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.223006  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.137090 15469 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:11.230996  9089 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.157421 15104 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:11.188810 14986 raft_consensus.cc:2749] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.208881 14800 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:11.237855  9119 raft_consensus.cc:2468] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:06:11.156836 15341 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:11.148614 14809 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:11.150472 15010 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.249624  9235 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:11.251973  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:11.209231 14914 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:06:11.236537 15104 raft_consensus.cc:493] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.248750 15250 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:06:11.259469  9859 raft_consensus.cc:2393] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 19: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:06:11.261297  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 45416.9us
I20260709 13:06:11.243893 14800 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:06:11.267279  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 117733us
I20260709 13:06:11.250177 14809 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
W20260709 13:06:11.274277  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 119008us
I20260709 13:06:11.277518  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.260841 15262 tablet_bootstrap.cc:700] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Time spent opening tablet: real 0.170s	user 0.003s	sys 0.000s
I20260709 13:06:11.279112  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:11.281085  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.262660 15104 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.280731  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.283422 14809 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.284775 14809 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:11.215106 15439 consensus_queue.cc:260] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.292186  9014 catalog_manager.cc:5697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 15. New cstate: current_term: 15 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.294240  9014 catalog_manager.cc:5697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:11.295897 15219 raft_consensus.cc:2804] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:11.298164 15303 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.299026 15463 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.310528 15334 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:11.314818  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:11.315652  9856 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 16: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:11.319262  9164 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:11.320186 15448 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:11.320581 15448 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:11.320973 15448 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.275677 14800 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:11.315444 15219 raft_consensus.cc:697] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:11.324529 15219 consensus_queue.cc:237] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.326567 15367 raft_consensus.cc:3055] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:11.327076 15367 raft_consensus.cc:740] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:11.328181 15367 consensus_queue.cc:260] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:11.329499  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 55881.2us
I20260709 13:06:11.329715 15367 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.330255  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.332281 15289 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:11.340863 15349 consensus_queue.cc:260] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.348435 15349 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:11.401836  9197 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:11.352016  9014 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 14, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 14 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:11.405750  9014 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:11.408481 15341 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 19. Current term is 20.
I20260709 13:06:11.354013 15444 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.357547  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:11.410655  9264 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.358371  9269 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:06:11.414520 15064 consensus_meta.cc:287] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.000s
I20260709 13:06:11.416353 15448 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.366521  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:11.360327 15290 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.417761  9193 raft_consensus.cc:606] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:11.331385  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.376449 15436 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.418311 15290 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:11.418928 15436 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.376024  9118 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:06:11.384250 14809 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:06:11.419936  9118 pending_rounds.cc:85] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 9
W20260709 13:06:11.384585  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 103593us
I20260709 13:06:11.420193  9118 pending_rounds.cc:107] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:06:11.420432  9118 raft_consensus.cc:2889] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:11.421171 15436 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.351485 15439 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:11.391081  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:11.421635  9089 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:11.422011  9114 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
I20260709 13:06:11.422179 15446 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:11.391775  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:11.422710 15446 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:11.423192 15446 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:11.424117  9089 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.426146 14914 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.428369  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:11.429003  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:11.429773  9269 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:06:11.431095 15303 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:11.431528 15303 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.431895 15303 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:06:11.432310  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 101035us
I20260709 13:06:11.432570  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:11.433269  9856 raft_consensus.cc:2393] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:11.435262 15104 raft_consensus.cc:515] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.436172  9162 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:11.437225 15432 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:11.437691 15432 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.438120 15432 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:11.431257  9089 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.440361 15104 leader_election.cc:290] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.442423 14914 raft_consensus.cc:2749] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:11.443732 15448 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.444329  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:11.444685  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:11.446631 15446 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.447901  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:11.348428  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.416203 15310 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.449754  9192 raft_consensus.cc:2368] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12.
I20260709 13:06:11.450241  9859 raft_consensus.cc:2368] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17.
I20260709 13:06:11.446890  9089 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.451052  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:11.451733  9269 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:06:11.452055 14914 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.328799 15010 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.422430 14809 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:11.453130 14809 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:11.454137  9826 leader_election.cc:400] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16.
I20260709 13:06:11.454061 14809 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.454393  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.454471  9826 leader_election.cc:403] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:06:11.457175 15481 raft_consensus.cc:3060] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:11.459292 15310 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.463641 15250 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.464326  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:11.464913  9269 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:06:11.465204  9089 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.466847 15250 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:11.444288 14800 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:11.461686  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 56919.3us
I20260709 13:06:11.474922  9859 raft_consensus.cc:1240] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 12. Current term is 13. Ops: []
I20260709 13:06:11.392890  9263 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:06:11.460114 15290 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:11.479506  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.479722  9162 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:11.480994  9192 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting Update request from peer 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:11.482982 15341 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:11.483557  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:11.483954 15341 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:11.484555  9189 raft_consensus.cc:2368] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:11.485283 15341 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.489207 15481 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16.
I20260709 13:06:11.489786  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:11.490285  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:11.493683  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:11.494334  9858 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:11.495591 14914 raft_consensus.cc:493] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:11.496204 14914 raft_consensus.cc:515] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.498227 15468 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:11.498292 14914 leader_election.cc:290] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.498492  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:11.499029 15468 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.499399  9118 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:06:11.501048 15468 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.501787  9858 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 20.
I20260709 13:06:11.503280  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:11.510373 14800 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.505048  9164 leader_election.cc:400] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17.
I20260709 13:06:11.516208  9164 leader_election.cc:403] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:11.516285  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:11.518231 15064 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.521294  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.522580  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:11.523962  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.525453  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b7da62e1fea4b79963197feee655d26" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:11.530546 15446 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:11.532378  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.532956  9122 raft_consensus.cc:3060] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:11.526168  9014 catalog_manager.cc:5697] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:11.537698  9014 catalog_manager.cc:5697] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 11, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 11 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:11.504891 15469 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.549655  9122 raft_consensus.cc:2468] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:11.506011 15010 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:11.549213  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:11.550814  9193 raft_consensus.cc:606] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:11.550609 15010 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.551184  9193 raft_consensus.cc:614] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:11.551494 15469 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.551657  9190 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:11.552688  9860 raft_consensus.cc:2368] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12.
I20260709 13:06:11.552810 15010 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:11.554761  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:11.555480  9122 raft_consensus.cc:2393] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 10: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:11.507728 15290 raft_consensus.cc:993] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:11.511260 15080 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:11.512441  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.504058  9119 pending_rounds.cc:85] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 10
I20260709 13:06:11.566339  9119 pending_rounds.cc:107] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:06:11.566640  9119 raft_consensus.cc:2889] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:11.545833  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:11.549863  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:11.568255  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.568610  9189 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 20.
I20260709 13:06:11.568830  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.569232  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:11.556445  9264 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:11.562695  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:11.563081  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaf63fb358d848ad991a929b2e8e0168" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:11.563091  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:11.570849  9271 raft_consensus.cc:3060] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.573063  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:11.561239 15289 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.578083 15289 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.580158 15289 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.582198  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709565930ea044f08322d94e29d83adf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.583048  9122 raft_consensus.cc:2368] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16.
I20260709 13:06:11.584041  9192 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:11.588279  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.582322  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.592705  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.594861 15290 raft_consensus.cc:993] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:11.595377 15290 raft_consensus.cc:1081] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:11.587066  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:11.597894  9238 leader_election.cc:400] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12.
I20260709 13:06:11.598191  9238 leader_election.cc:403] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:11.598667  9266 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:06:11.599496  9266 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.600000  9266 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:11.600863 15444 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:06:11.602059  9238 leader_election.cc:400] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:11.602377  9238 leader_election.cc:403] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:06:11.602345  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.610738  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:11.613122  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.613205  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.605258  9828 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:11.625417  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:11.626410  9860 raft_consensus.cc:2368] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:11.620416  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:11.590845 14800 raft_consensus.cc:1081] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:11.637868  9269 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:06:11.641057 15222 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.641152  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:11.629062  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.641475 15458 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:11.654321  9196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:11.615306 15468 raft_consensus.cc:2749] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:11.644065  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:11.662774  9858 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:11.663388  9196 raft_consensus.cc:2393] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:11.645504  9193 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:11.646385 15463 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:11.647035  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.623349 15483 raft_consensus.cc:2804] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:11.664366  9856 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:11.634744  9089 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:11.648088 15461 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.625010 15482 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:11.612185 14809 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:11.657619  9116 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:11.659205  9269 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:11.659454  9238 leader_election.cc:400] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12.
I20260709 13:06:11.642625 15435 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:11.586263 15303 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.643015 15221 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:11.645042  9190 raft_consensus.cc:1275] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:11.684813  9193 raft_consensus.cc:606] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:11.685267  9193 raft_consensus.cc:614] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:11.660933  9266 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.627831 14914 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:11.680140 15461 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.683265  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.654575 15446 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 11. Current term is 12.
I20260709 13:06:11.683485 15495 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:11.718470  9272 raft_consensus.cc:2368] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18.
I20260709 13:06:11.685806 15270 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:11.688218 15483 raft_consensus.cc:697] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:11.663763 15222 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.716332  9266 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:11.707263 14986 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:11.660169 15458 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:11.728386  9271 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:11.728317  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:11.743558  9123 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:06:11.690405  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55249.7us
I20260709 13:06:11.725723 15495 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:11.746492  9825 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.747114  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.572621  9014 catalog_manager.cc:5697] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 13, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 13 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:11.705843 15221 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:11.766036  9120 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:06:11.747862  9089 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:11.741122 15458 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:11.740500 15064 raft_consensus.cc:2749] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17.
I20260709 13:06:11.768373  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:11.769194 15270 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.781031  9189 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 10.
I20260709 13:06:11.774295 14809 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:06:11.796072  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 112869us
I20260709 13:06:11.804942  9238 leader_election.cc:400] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
W20260709 13:06:11.822288  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 75841.6us
I20260709 13:06:11.775362 15432 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:11.846840  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 58447.1us
W20260709 13:06:11.849251 15484 log.cc:927] Time spent T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 13:06:11.849900 15494 log.cc:927] Time spent T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:06:11.850087 15221 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.851660 15461 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.852624  9825 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.863858  9195 raft_consensus.cc:2468] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
W20260709 13:06:11.866758  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 48477.2us
I20260709 13:06:11.867210  9825 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.867702  9238 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.866235 15270 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:11.870319  9825 leader_election.cc:400] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16.
I20260709 13:06:11.874931  9238 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:11.880271 15501 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:11.882266 15501 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:11.882723 15501 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:11.883152 15501 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:11.877730  9826 leader_election.cc:304] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:11.901350  9825 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:11.907208  9826 leader_election.cc:400] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18.
I20260709 13:06:11.903406 15303 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:11.876638 15447 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:11.915274 15270 raft_consensus.cc:2804] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:11.917258 15270 raft_consensus.cc:697] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:11.918045 15432 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.918143 15270 consensus_queue.cc:237] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.852437 15483 consensus_queue.cc:237] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.892087 15495 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.926183 15222 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:11.922513 15377 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:11.926800 15502 raft_consensus.cc:2749] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:11.959226  9272 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:06:11.965267  9272 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:11.965642  9272 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:11.967226 15178 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:11.967775 15178 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:11.968698 15178 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:11.970009 15178 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:12.063097  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:12.092079  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:12.093343  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35f9598c50046539a7f41287876135d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:12.035830 15377 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:12.112911  9123 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
I20260709 13:06:12.115219  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:12.115919  9266 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 9.
I20260709 13:06:12.114387  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:12.117096  9269 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:06:12.121076  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:12.124526  9269 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:12.123842  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:12.126804  9266 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:06:12.131605  9268 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:12.080139 15501 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.155368  9198 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:12.155855  9198 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:12.183036  9269 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:06:12.161424  9272 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:12.187999  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 282897us
I20260709 13:06:12.191164  9272 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:12.193343  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:12.203474  9194 raft_consensus.cc:3055] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:12.186852  9198 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.203951  9194 raft_consensus.cc:740] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:12.215426  9194 consensus_queue.cc:260] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:12.244664  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 343379us
I20260709 13:06:12.245954  9825 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:12.199965 15377 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.247732 15458 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.250361 15470 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:12.259402 15501 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:12.269954  9193 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:12.271312 15498 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:12.272503 15498 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:12.272915 15498 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:12.273782 15498 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.275345 15498 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:12.276399  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:12.277086  9120 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:12.282610 15341 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.283196 15341 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.284237 15458 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:12.287063  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:12.288683 15341 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:12.291712  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:12.292624  9195 raft_consensus.cc:2393] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:12.295903 15511 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.296523 15511 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.298233  9198 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:12.303387 15411 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:12.303843 15411 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:12.304713 15411 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.305790 15411 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:12.298981 15511 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:12.308513  9194 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:12.316089  9162 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:12.317530  9162 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:12.318476 15262 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=3 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:06:12.319298 15290 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:12.331157  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:12.331072  9117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:12.329660  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:12.337561  9116 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 10.
I20260709 13:06:12.337697  9238 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:12.329293  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:12.338492  9194 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:06:12.338577 15503 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:06:12.331522  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c07692445f814d5cab7dd13d622f9b9c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:12.339164 15503 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:12.319613  9162 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:12.340472  9238 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:12.340827  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:12.328820  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:12.341585 15310 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:06:12.341763  9264 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:06:12.342034 15310 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.342164 15446 raft_consensus.cc:2804] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:12.330613 15501 consensus_queue.cc:1048] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:06:12.342701 15080 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:12.342759 15446 raft_consensus.cc:697] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:12.343230 15080 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.343639 15080 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:12.343911 15446 consensus_queue.cc:237] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.342483 15310 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:12.339609 15503 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:06:12.338811  9119 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
W20260709 13:06:12.351599  9237 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.320158 15222 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:12.354423  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:12.355312 15431 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:12.339552  9122 raft_consensus.cc:3060] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:12.367161  9162 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:12.337709  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:12.366698  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:12.380805  9089 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:12.386974  9089 leader_election.cc:304] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:12.355713 15431 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.319659 15262 tablet_bootstrap.cc:492] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Bootstrap complete.
I20260709 13:06:12.352905 15310 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.401718  9189 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:06:12.404186 14809 raft_consensus.cc:2749] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:12.319949 15290 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 17 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:12.363940 15222 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:12.413465 15222 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.391767 15468 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:12.399075 15511 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:12.391511 15431 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:12.408408 15262 ts_tablet_manager.cc:1403] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Time spent bootstrapping tablet: real 1.389s	user 0.060s	sys 0.032s
I20260709 13:06:12.417317  9120 raft_consensus.cc:2468] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 18.
I20260709 13:06:12.428984  9198 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
I20260709 13:06:12.427278  9122 raft_consensus.cc:2468] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 12.
I20260709 13:06:12.409037 15221 raft_consensus.cc:2804] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:12.424443 15250 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:06:12.449827 15290 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.444451 15221 raft_consensus.cc:697] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:12.456641 15221 consensus_queue.cc:237] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.466568  9264 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:06:12.466625  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:12.469558  9120 raft_consensus.cc:1240] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:06:12.428061 15511 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.477356  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:12.478641  9122 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:06:12.485384  9268 raft_consensus.cc:1240] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:12.479813 15511 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:12.494257  9272 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:06:12.495565  9087 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:12.482908 15470 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:12.476703 14914 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:06:12.484786 15498 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:06:12.506970  9272 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:12.502696 14809 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:06:12.531814  9827 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:06:12.547849  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 49947.5us
I20260709 13:06:12.523929 14809 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:12.546986 15250 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.532311 15310 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:12.569371 15250 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.569906  9161 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:12.570994 15497 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:12.571465 15497 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.571863 15497 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:12.573763 15470 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.574450 15080 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.581728  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:12.582396  9120 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:12.562206 15010 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:12.587558 15010 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.590567 15503 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.593675 15517 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:12.595605  9119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:12.596261  9119 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:12.564400 14809 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.584245 15439 raft_consensus.cc:493] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:12.585554 15497 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.586890  9853 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:06:12.612792  9856 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:12.616518  9235 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:12.613406  9856 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:06:12.563414  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:12.617349  9122 raft_consensus.cc:3060] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:12.623242 15497 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:12.594326 15517 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.624168 15010 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:12.624792 15289 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:12.625560 15517 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:12.625432 15289 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:12.626710 15519 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:06:12.628412 15289 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:12.592723 15503 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:12.651153  9121 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:06:12.651386  9164 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:12.647527  9195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:12.652756  9195 raft_consensus.cc:2368] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14.
I20260709 13:06:12.656482  9119 raft_consensus.cc:2468] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:12.656879  9195 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:12.662604  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:12.664083  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:12.676695  9271 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:06:12.676776  9122 raft_consensus.cc:2468] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:06:12.665460  9114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:12.677656  9114 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:12.665894  9238 leader_election.cc:400] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14.
I20260709 13:06:12.663218  9120 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:06:12.667618  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:12.678215  9238 leader_election.cc:403] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:12.668079  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a08146adfeb9417ca7067de38da555b0" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:12.610524 15250 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:12.670732  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:12.670853  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:12.681242  9198 raft_consensus.cc:2368] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
W20260709 13:06:12.672717  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 56280.7us
I20260709 13:06:12.680250  9161 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:12.682632  9235 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:06:12.671646 15524 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.102s	user 0.003s	sys 0.000s
I20260709 13:06:12.691605  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "470326df6cae476c90888533e504e6ec" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:12.688432 15499 raft_consensus.cc:3060] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:06:12.693953  9087 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:12.641039 15439 raft_consensus.cc:515] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.700822  9238 leader_election.cc:400] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
I20260709 13:06:12.702677  9238 leader_election.cc:403] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:06:12.679893 15503 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:06:12.712951  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 204074us
I20260709 13:06:12.667625 15270 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:06:12.687831 15104 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:12.695315 15010 raft_consensus.cc:2804] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:12.695184 15448 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:06:12.724968  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:12.726365  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49752620cbbf430991915a2400f71286" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:12.727131  9266 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:12.702719 15250 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:12.680748 15511 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.733163  9861 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:12.711068 15080 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:12.740720  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:12.742378  9266 raft_consensus.cc:2468] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:06:12.743359  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc89bbd2f2934bbeaf33619d3d23adfc" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:12.749936  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:12.721880 15438 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:06:12.765512  9855 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:12.722054 15503 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:12.769667  9860 raft_consensus.cc:1240] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 16. Current term is 17. Ops: []
I20260709 13:06:12.731549 15250 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:12.775591  9828 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:12.747170 15448 raft_consensus.cc:697] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:12.729290 15104 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.744380 15436 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:06:12.780548 15250 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:12.782060 15463 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:12.808359  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:12.800482  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 78140.1us
I20260709 13:06:12.802685 15439 leader_election.cc:290] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:12.810931  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9a6ffaa328644f4bd7f129f8fa22875" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:12.817546  9198 raft_consensus.cc:1275] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:06:12.820999 15532 log.cc:927] Time spent T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:06:12.821918 15104 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:12.824143 15357 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:12.826604  9235 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:12.828150 15289 raft_consensus.cc:2804] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:12.828747 15289 raft_consensus.cc:697] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:12.829756 15289 consensus_queue.cc:237] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.831197  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:12.832758  9121 raft_consensus.cc:1275] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:06:12.832963  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:12.833823  9120 raft_consensus.cc:2368] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
I20260709 13:06:12.838383 15448 consensus_queue.cc:237] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.840408  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:12.842512 15503 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.846576  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:12.848522 15538 consensus_queue.cc:1048] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:12.852221  9235 leader_election.cc:400] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 10 election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
I20260709 13:06:12.847378  9268 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:12.852524  9162 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:12.855090  9162 leader_election.cc:304] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:12.811683 15535 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:12.867769  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:12.873673  9194 raft_consensus.cc:2393] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 17: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:12.875097  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:12.861794 15530 raft_consensus.cc:2804] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:12.864313 15529 raft_consensus.cc:2749] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:12.856033 15438 raft_consensus.cc:2749] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 10. Current term is 11.
W20260709 13:06:12.871999 15541 log.cc:927] Time spent T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:06:12.861690 15010 raft_consensus.cc:697] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 18 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:12.874894 15262 raft_consensus.cc:359] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: NON_VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: true } }
I20260709 13:06:12.866606 15535 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.895982  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:12.863374 15447 consensus_queue.cc:1048] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
W20260709 13:06:12.941550  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 45679.1us
W20260709 13:06:12.962031  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 57678.4us
I20260709 13:06:12.969996 15262 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Initialized, Role: FOLLOWER
I20260709 13:06:12.970780 15262 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:12.972330 15533 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:06:12.974681  9114 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
W20260709 13:06:12.975195 15431 consensus_meta.cc:287] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.000s
I20260709 13:06:12.975663 15431 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.975786 15010 consensus_queue.cc:237] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:12.910564 15530 raft_consensus.cc:697] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:12.910935 14914 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:12.991995  9857 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:12.996064  9190 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:06:13.001816  9271 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 11.
I20260709 13:06:12.965201 15104 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:12.986539 15511 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:13.030507 15544 log.cc:927] Time spent T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:06:13.033041 15431 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:13.033978 15530 consensus_queue.cc:237] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:12.992522 15499 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14.
I20260709 13:06:13.041687 15432 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
W20260709 13:06:13.054684  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 91870.9us
I20260709 13:06:13.061378  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:13.062567  9264 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:13.094053  9269 raft_consensus.cc:2468] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:06:13.051971 15310 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:13.007954 14914 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:13.135504  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 117392us
I20260709 13:06:13.106058 15349 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
W20260709 13:06:13.203830 15551 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 13:06:13.206386 15545 log.cc:927] Time spent T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
W20260709 13:06:13.208259 15515 log.cc:927] Time spent T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.819s	user 0.005s	sys 0.007s
I20260709 13:06:13.105670 15310 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:13.219990  9862 raft_consensus.cc:1217] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 19.12->[19.13-19.13]   Dedup: 19.13->[]
W20260709 13:06:13.228717  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 143036us
I20260709 13:06:13.209708 15436 raft_consensus.cc:493] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:13.220770  9862 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 6 to 13, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:13.215637 15349 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.288682 15548 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.322639 15262 ts_tablet_manager.cc:1434] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Time spent starting tablet: real 0.882s	user 0.016s	sys 0.006s
W20260709 13:06:13.323031 15262 ts_tablet_manager.cc:1467] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Tablet startup took 891ms
I20260709 13:06:13.239223 15436 raft_consensus.cc:515] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.317495 14914 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:13.312443 15535 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:13.336731  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:13.336856  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 115299us
I20260709 13:06:13.340704  9269 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:06:13.336211 14986 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:13.349804 14986 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.289166 15549 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.129s
I20260709 13:06:13.315578 15548 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:13.349810 15250 consensus_meta.cc:287] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.002s
I20260709 13:06:13.358665 15502 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:13.375083 15502 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:13.323383 15262 ts_tablet_manager.cc:1469] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Trace:
0709 13:06:04.858031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:04.874912 (+ 16881us) service_pool.cc:224] Handling call
0709 13:06:05.328868 (+453956us) ts_tablet_manager.cc:934] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: Initiating tablet copy from peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
0709 13:06:08.320250 (+2991382us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:06:08.824846 (+504596us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:09.037533 (+212687us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:09.058167 (+ 20634us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 13:06:09.494303 (+436136us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:06:09.495072 (+   769us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:09.495698 (+   626us) ts_tablet_manager.cc:989] Replica registered.
0709 13:06:09.511794 (+ 16096us) inbound_call.cc:177] Queueing success response
0709 13:06:11.018754 (+1506960us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:06:12.443339 (+1424585us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:06:12.571841 (+128502us) tablet_replica.cc:261] Starting instrumentation
0709 13:06:12.572541 (+   700us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":19373,"mutex_wait_us":28312,"raft.queue_time_us":14976,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":2924,"tablet-copy.queue_time_us":247892,"tablet-download-pool.queue_time_us":115287,"tablet-download-pool.run_cpu_time_us":5605,"tablet-download-pool.run_wall_time_us":803761,"thread_start_us":297156,"threads_started":3}
I20260709 13:06:13.382922 15436 leader_election.cc:290] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:13.388929 14986 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:13.380101 15250 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.392010  9122 raft_consensus.cc:2368] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14.
I20260709 13:06:13.391507 15250 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:13.366219 15555 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:13.393931  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:13.384299 15310 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:13.418043 15349 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:13.419974 15555 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.422339 15555 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:13.426594 15548 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:13.417980  9162 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:13.430244 15432 raft_consensus.cc:2804] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:13.430773 15432 raft_consensus.cc:697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:13.431562 15432 consensus_queue.cc:237] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:13.434459  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:13.429904  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:13.446000  9858 raft_consensus.cc:3055] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:13.448508  9858 raft_consensus.cc:740] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:13.447523  9861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:13.435271  9271 raft_consensus.cc:2393] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:13.450469  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:13.452139  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:13.453267 15538 raft_consensus.cc:493] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:13.453791 15538 raft_consensus.cc:515] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.454289  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:13.455858 15502 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:13.457465  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:13.458549  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:13.460738  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:13.445616 15535 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:13.461833 15535 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.462157 15535 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:13.459550  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:13.464596  9859 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:13.467201  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:13.471005  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:13.471801  9120 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
I20260709 13:06:13.449373  9858 consensus_queue.cc:260] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.472678  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:13.478711  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:13.479529  9123 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:06:13.479476  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:13.473788  9268 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:06:13.480603  9264 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:13.473425  9858 raft_consensus.cc:3060] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:13.487233  9268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:13.489185  9268 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 15.
I20260709 13:06:13.485463  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b71b09f6834fbbbd500874ff1e14b5" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:13.495661  9858 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:06:13.497152  9114 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:13.499583  9197 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:13.497872  9117 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:13.502743  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:13.503439  9271 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:06:13.506170  9858 pending_rounds.cc:85] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 11
I20260709 13:06:13.506678  9858 pending_rounds.cc:107] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:06:13.507067  9858 raft_consensus.cc:2889] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:13.511972  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:13.496461 15495 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:06:13.518425 15561 raft_consensus.cc:493] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:13.519634  9116 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:13.522169  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:13.524982  9087 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:13.525274  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:13.525986  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:13.527050  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:13.528452 14914 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:13.529558  9192 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:06:13.529708 15458 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:13.530078 15458 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.530409 15458 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:13.535951  9192 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:06:13.572721  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 52747.2us
I20260709 13:06:13.496240 15517 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:13.573765  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:13.576040  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:13.489693 15538 leader_election.cc:290] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:13.585680  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:13.585937  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48788.7us
I20260709 13:06:13.591719  9269 raft_consensus.cc:1275] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:13.594310  9860 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40108 (request call id 1870) took 1076 ms. Trace:
I20260709 13:06:13.594702  9860 rpcz_store.cc:276] 0709 13:06:12.514439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.514547 (+   108us) service_pool.cc:224] Handling call
0709 13:06:12.514846 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:06:12.772454 (+257608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.590736 (+818282us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b5400660b54. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:13.590906 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":579232512}
I20260709 13:06:13.598995  9271 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:06:13.626324 15523 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:13.636080 15523 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:13.636524 15523 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:13.519038 15561 raft_consensus.cc:515] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.626873 15542 consensus_queue.cc:1059] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:06:13.640120 15542 raft_consensus.cc:3055] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:13.640535 15542 raft_consensus.cc:740] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:13.641469 15542 consensus_queue.cc:260] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.642781 15542 raft_consensus.cc:3060] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:13.580322 15517 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:13.639386 15561 leader_election.cc:290] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:13.670595  9269 raft_consensus.cc:1240] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:13.671442  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 144441us
I20260709 13:06:13.660140 15411 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:13.728160  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 63517.8us
I20260709 13:06:13.730937  9118 raft_consensus.cc:1275] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:06:13.583076 15495 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:06:13.740082  9281 tablet_copy_service.cc:342] P 1efabcb1befc400aadf101e8276fc1aa: Request end of tablet copy session 92919b70789041c8a0de0b3525154fd1-685fa3ec557f4f3db02ec6ccce1d6f3e received from {username='slave'} at 127.0.0.1:38650
I20260709 13:06:13.740744  9281 tablet_copy_service.cc:434] P 1efabcb1befc400aadf101e8276fc1aa: ending tablet copy session 92919b70789041c8a0de0b3525154fd1-685fa3ec557f4f3db02ec6ccce1d6f3e on tablet 685fa3ec557f4f3db02ec6ccce1d6f3e with peer 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:13.675309 15411 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.750221 15411 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:13.751204  9190 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38874 (request call id 2605) took 1170 ms. Trace:
I20260709 13:06:13.751441 15538 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:13.751633  9190 rpcz_store.cc:276] 0709 13:06:12.580461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.582991 (+  2530us) service_pool.cc:224] Handling call
0709 13:06:12.583309 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.996611 (+413302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.750916 (+754305us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b5400217354. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e4e01 00007f76566e383e 00007f765b6a8419 00007f7654c1fbd4 00007f7654c1fb62 00007f7654c1fa91 00007f7654c1fa0c 00007f7654c1ecc2 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:13.751120 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948510464}
I20260709 13:06:13.666759 15517 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:13.755697  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:13.781877  9860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:13.782444  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fdaa422821a46b8bc73801d1abfb3c0" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:13.784415  9121 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:06:13.793042  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:13.778616 15495 consensus_queue.cc:1243] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer 94504d29c6744f38bb06943479df9466 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:06:13.794747  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:13.796051 15501 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:06:13.797720 15552 log.cc:927] Time spent T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.482s	user 0.002s	sys 0.000s
I20260709 13:06:13.806770  9266 raft_consensus.cc:1240] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 18 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:13.818303  9238 leader_election.cc:304] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:13.831908 15569 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:06:13.777760 15539 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:06:13.834467 15562 raft_consensus.cc:2749] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:13.844010  9161 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:13.815548 15529 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:13.851709  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:13.862839  9859 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 14.
I20260709 13:06:13.864136  9089 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:13.865240  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:13.865716  9121 raft_consensus.cc:1275] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:13.815786 15439 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:06:13.873589 15538 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:13.865600 15250 consensus_queue.cc:1059] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:06:13.877689 15517 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.883666 15582 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:13.884480 15582 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:13.885442 15582 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:13.854310 15529 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:13.848549 15481 consensus_queue.cc:1048] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:06:13.906764  9855 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1491) took 1005 ms. Trace:
W20260709 13:06:13.912504  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 93856.1us
I20260709 13:06:13.907353  9855 rpcz_store.cc:276] 0709 13:06:12.897923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.898101 (+   178us) service_pool.cc:224] Handling call
0709 13:06:12.898582 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:06:12.899017 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:06:12.899056 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:06:12.960483 (+ 61427us) log.cc:844] Serialized 26 byte log entry
0709 13:06:13.105699 (+145216us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:06:13.105743 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:06:13.105806 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:06:13.797037 (+691231us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b4400022b4c. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f765351dc85 00007f765351e6a2 00007f76562901db 00007f765628baaf 00007f765627f86e 00007f765627ebb6 00007f765627e7d8 00007f7656288221 00007f76562881d9 00007f7656288169
0709 13:06:13.893804 (+ 96767us) raft_consensus.cc:1685] finished
0709 13:06:13.893833 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:06:13.903236 (+  9403us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6177,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":31678,"spinlock_wait_cycles":3998464,"thread_start_us":112400,"threads_started":1,"wal-append.queue_time_us":240924}
I20260709 13:06:13.911818 15529 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:13.927563  9264 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:13.891706 15535 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:13.938333 15521 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:06:13.962574  9235 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:13.928872 15357 consensus_queue.cc:1048] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:06:13.965010  9194 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 16: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:13.932049 15495 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:13.939287 15482 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 13:06:13.983093  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 53296.3us
I20260709 13:06:13.931769 15523 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:13.979004 15574 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.397s	user 0.000s	sys 0.003s
I20260709 13:06:13.974334 15538 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:13.973429 15495 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.003912 15458 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.006069 15458 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:14.009117 15584 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:06:14.019869  9122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43780 (request call id 2930) took 1168 ms. Trace:
I20260709 13:06:14.020459  9122 rpcz_store.cc:276] 0709 13:06:12.830781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:12.830942 (+   161us) service_pool.cc:224] Handling call
0709 13:06:13.998957 (+1168015us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b5400356bd4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566e82fa 00007f765b6a9b17 00007f7654c246d4 00007f7654c24662 00007f7654c24591 00007f7654c2450c 00007f7654c237c2 00007f7654a1f48c 00007f7654a1e8c6 00007f7654a1e26f
0709 13:06:13.999083 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287972224}
I20260709 13:06:13.978977 15502 consensus_queue.cc:1048] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:06:13.979247 15349 consensus_queue.cc:1059] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
W20260709 13:06:14.049438  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 75004us
I20260709 13:06:14.061228  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:14.064779  9271 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:14.001773 15521 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:14.048898 14809 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:14.048007 15539 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:14.049546 15503 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:14.079126  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
W20260709 13:06:14.081797  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 217775us
I20260709 13:06:14.069048 15289 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:14.091634  9271 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 16.
I20260709 13:06:14.071446 15517 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:14.082742 15579 log.cc:927] Time spent T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:06:14.108330 15592 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:14.109155 15592 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.081748 15503 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.111792 15503 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:06:14.093070 15539 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.116242 15539 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.096297 15289 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:14.092749 15588 log.cc:927] Time spent T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:06:14.129693  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:14.147574  9267 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:14.149366  9271 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
W20260709 13:06:14.149914 15597 log.cc:927] Time spent T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:06:14.163545  9268 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:06:14.072834 15521 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:14.166014  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 73295.2us
I20260709 13:06:14.127079 15289 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.151149 15535 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:14.168251  9267 raft_consensus.cc:1275] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:14.168049  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:14.122469 15529 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.149410 15592 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:14.189525  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:14.190656  9267 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:14.198407 15595 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.199102 15595 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:14.204146 15585 log.cc:927] Time spent T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:06:14.206481 15594 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:14.207157 15594 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.209692 15594 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.210325 15495 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.211900 15529 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:14.213033  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:14.213966  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:14.214560  9855 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 18.
I20260709 13:06:14.222820  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:14.225301 15349 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:14.221168 15290 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:06:14.225812 15349 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:14.226256 15349 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:14.227420 15591 consensus_queue.cc:1048] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:14.218727 15523 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.233886 15349 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.237939  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:14.246145  9855 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 18.
I20260709 13:06:14.244993 15010 raft_consensus.cc:493] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:14.247457  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:14.247473 15010 raft_consensus.cc:515] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.249022 15349 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.249686 15349 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:14.250203 15010 leader_election.cc:290] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.250623 15349 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:14.252206 15349 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.253914 14809 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:14.254912 14809 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:14.259404 15594 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.260015 15594 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.263511 14809 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.266062 14809 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.267131  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:14.272598  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:14.273600  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:14.274350  9118 raft_consensus.cc:3055] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:14.274765  9118 raft_consensus.cc:740] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:14.275568  9118 consensus_queue.cc:260] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.280357  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:14.281208  9123 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:14.281550  9120 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:06:14.284116  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:14.285058 15547 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:14.285660 15547 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.286237  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:14.287032  9189 raft_consensus.cc:2393] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 16: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:14.289263  9189 raft_consensus.cc:1240] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:06:14.290514  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.291391  9192 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 18: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:14.292764  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.298563  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:14.299428  9189 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 18: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:14.299708  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:14.300127 15521 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.301172 15498 raft_consensus.cc:3055] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:14.301568 15498 raft_consensus.cc:740] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:14.290675 15547 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:14.303156  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
W20260709 13:06:14.304219 15577 log.cc:927] Time spent T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: Append to log took a long time: real 0.600s	user 0.004s	sys 0.000s
I20260709 13:06:14.306015 15498 consensus_queue.cc:260] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.307272 15498 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:06:14.307917  9828 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:14.311504  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:14.315302  9235 leader_election.cc:400] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14.
I20260709 13:06:14.315620  9235 leader_election.cc:403] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:14.317771 15605 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 13. Current term is 14.
I20260709 13:06:14.336252  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:14.337616  9089 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:14.339160 15349 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:14.340240  9121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:14.339370  9161 leader_election.cc:304] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:14.346138 15443 raft_consensus.cc:493] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:14.346721 15443 raft_consensus.cc:515] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.347697 15538 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.348294 15538 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.349606 15461 raft_consensus.cc:2749] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:14.350912 15538 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:14.353936  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:14.354627  9116 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:14.359242  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:14.363708  9235 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:14.367205 14986 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:14.367692 14986 raft_consensus.cc:493] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.368111 14986 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:14.371644  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.373728  9192 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:06:14.374327  9118 raft_consensus.cc:3060] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:14.377027  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:14.377737  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:14.378217  9189 raft_consensus.cc:2368] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18.
I20260709 13:06:14.378690 15517 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:14.379223 15517 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:14.380196 15517 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.380851  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.383709  9087 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:14.384549  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:14.385651 15593 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:14.386320 15593 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:14.387197 15593 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.390385  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:14.391178  9862 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
I20260709 13:06:14.387367 15595 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.392688 15461 consensus_queue.cc:1059] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:06:14.394837 15221 raft_consensus.cc:3055] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:14.395227 15221 raft_consensus.cc:740] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:14.396076  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:14.396091 15221 consensus_queue.cc:260] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.397894  9161 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:14.398569 15221 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:14.398711 15522 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:14.353135 15443 leader_election.cc:290] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.405185 15522 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:14.406162 15522 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.418974  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:14.420719  9858 raft_consensus.cc:1275] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:06:14.421178  9858 pending_rounds.cc:85] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 13
I20260709 13:06:14.421423  9858 pending_rounds.cc:107] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:06:14.421694  9858 raft_consensus.cc:2889] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:14.423185  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:14.424714  9858 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:14.427835  9123 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:06:14.429677  9826 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:14.430660  9123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e874ff0724409b88c5ec377b73a06a" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:14.433324  9087 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:14.434372 15498 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:14.437291  9116 raft_consensus.cc:1240] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:06:14.437960  9123 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:14.442044  9162 leader_election.cc:304] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:14.442445 15594 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:14.443073 15497 raft_consensus.cc:2749] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:14.443681  9272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:14.444562  9162 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:14.445920 15595 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:06:14.447654  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.450098  9164 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:14.458760  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.460943  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.450013  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:14.450600  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.465880  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.467949  9189 raft_consensus.cc:1217] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Deduplicated request from leader. Original: 11.9->[13.10-14.11]   Dedup: 13.10->[14.11-14.11]
I20260709 13:06:14.468535  9189 raft_consensus.cc:1240] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 14. Current term is 15. Ops: [13.10-14.11]
I20260709 13:06:14.471330  9271 raft_consensus.cc:1275] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:06:14.474215  9271 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:06:14.454615 15606 log.cc:927] Time spent T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:06:14.477955  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:14.479592 15349 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:14.483425  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:14.484109  9264 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:14.494786 15339 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:14.495391 15339 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:06:14.495712  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 66112.5us
I20260709 13:06:14.498566 15568 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:14.499043 15568 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:14.499416 15568 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:14.495831 15339 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:14.506474  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 79911.5us
I20260709 13:06:14.497798  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:14.507279  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:14.507812  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 59404.1us
I20260709 13:06:14.508045 15547 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:14.508669 15511 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:14.509078 15511 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:14.509404 15511 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:14.520480  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 68403.6us
I20260709 13:06:14.521256 15517 consensus_queue.cc:1059] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:14.524924 15547 raft_consensus.cc:3055] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:06:14.524178  9021 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 6 to 14, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 14 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.525394 15547 raft_consensus.cc:740] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:14.517747 15511 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.526346  9021 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:14.526429 15547 consensus_queue.cc:260] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.534147  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:14.514354 15568 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:14.549456  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55324.9us
I20260709 13:06:14.543426  9828 leader_election.cc:400] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Vote denied by peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417) with higher term. Message: Invalid argument: T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18.
I20260709 13:06:14.542640  9264 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
W20260709 13:06:14.543906  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 62150.6us
I20260709 13:06:14.544278  9118 raft_consensus.cc:1275] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:06:14.552637  9118 pending_rounds.cc:85] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Aborting all ops after (but not including) 10
I20260709 13:06:14.552959  9118 pending_rounds.cc:107] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:06:14.553264  9118 raft_consensus.cc:2889] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:14.560896  9162 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:14.562942  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbfd5d6417134672a62b2757b6d55dff" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:14.563253  9021 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:14.565649  9161 leader_election.cc:304] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:14.574915  9021 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 9 to 10, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 10 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:14.563455 15548 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:06:14.563845 15511 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:14.580797 15443 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:14.588160 15339 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.590375 15339 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:14.559934 15497 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:06:14.595504 15461 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:06:14.598064  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:14.599472  9853 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 19: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:14.600057 15432 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:06:14.583667 15568 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:14.614676  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 49091.1us
I20260709 13:06:14.639248 15547 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:14.532598 14986 raft_consensus.cc:515] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.589238  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:14.542640  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:14.662070  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 119514us
I20260709 13:06:14.664585  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:14.667707  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:14.674332  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 85235.3us
I20260709 13:06:14.677510  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:14.685815 15555 consensus_queue.cc:1048] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:06:14.675926 15497 consensus_queue.cc:1048] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:06:14.694027  9272 raft_consensus.cc:2368] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 14. Current term is 15.
I20260709 13:06:14.578236 15582 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:14.697853  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:14.701309  9192 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 19: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:14.662628 15529 raft_consensus.cc:2749] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:14.705343  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:14.706712  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:14.707137 15539 consensus_queue.cc:1048] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:14.692977 15432 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:14.709625 14986 leader_election.cc:290] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:14.694172 15566 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:14.704869 15582 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:14.713397  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:14.726863  9115 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:14.718545  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.667068 15562 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:14.722803  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:14.731509  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:14.731554 15499 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:14.750015  9855 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:06:14.757910 15609 log.cc:927] Time spent T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: Append to log took a long time: real 0.193s	user 0.003s	sys 0.000s
I20260709 13:06:14.742211 15505 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:14.760928 15582 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:14.742830  9238 leader_election.cc:304] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:14.734259 15568 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:06:14.805769  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 63018.5us
I20260709 13:06:14.787241  9115 raft_consensus.cc:2468] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 15.
I20260709 13:06:14.793689 15432 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:14.809041 15432 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:14.798787  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 67356.1us
I20260709 13:06:14.799352 15530 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:06:14.800392 15505 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.817831  9235 leader_election.cc:304] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:14.826723  9855 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:14.816059 15289 raft_consensus.cc:2749] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:14.874289  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 56542.3us
I20260709 13:06:14.796105 15594 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
W20260709 13:06:14.878649  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 55742.7us
I20260709 13:06:14.876844 15357 raft_consensus.cc:2804] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:14.882071  9162 leader_election.cc:400] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Vote denied by peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807) with higher term. Message: Invalid argument: T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 for earlier term 14. Current term is 15.
I20260709 13:06:14.774101  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:14.880419  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 66207.2us
I20260709 13:06:14.887096  9021 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:14.918915  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:14.922109  9197 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:14.900980 15289 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:14.934369  9021 catalog_manager.cc:5697] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 6 to 9, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 9 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.932762  9272 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:06:14.933410  9826 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:14.941350  9194 raft_consensus.cc:1240] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Rejecting Update request from peer ce51b61da22f40d4aa0204a6b1028f28 for earlier term 18. Current term is 19. Ops: []
I20260709 13:06:14.942320  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa32d6e0bb7b44ee9f664126e3ecdb72" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:14.946376  9021 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 9 to 14, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 14 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:14.944702 15289 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:14.943447 15411 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:14.949746 15505 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:14.934094 15357 raft_consensus.cc:697] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:14.969002 15357 consensus_queue.cc:237] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:14.972273  9014 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36626 (request call id 362) took 12765 ms (client timeout 15000 ms). Trace:
I20260709 13:06:14.987833  9021 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:15.060004  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:15.050000  9021 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 12 to 13, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 13 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:15.063181  9021 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 8 to 10, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 10 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:15.067710  9021 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 7 to 11, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 11 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:15.062597 15080 raft_consensus.cc:493] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:15.070818 15080 raft_consensus.cc:515] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.038230 15498 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:06:15.092379 15289 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:14.972846  9014 rpcz_store.cc:269] 0709 13:06:02.207081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:02.207362 (+   281us) service_pool.cc:224] Handling call
0709 13:06:14.972181 (+12764819us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:12.460054 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:12.460602 (+   548us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:12.460658 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:06:12.460679 (+    21us) tablet.cc:662] Decoding operations
0709 13:06:12.545224 (+ 84545us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:12.545331 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:12.545360 (+    29us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:06:12.632386 (+ 87026us) tablet.cc:701] Row locks acquired
0709 13:06:12.632426 (+    40us) write_op.cc:260] PREPARE: finished
0709 13:06:12.632606 (+   180us) write_op.cc:270] Start()
0709 13:06:12.632756 (+   150us) write_op.cc:276] Timestamp: P: 1783602372632541 usec, L: 0
0709 13:06:12.632794 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:06:12.719641 (+ 86847us) log.cc:844] Serialized 9947 byte log entry
0709 13:06:13.350474 (+630833us) op_driver.cc:464] REPLICATION: finished
0709 13:06:13.537668 (+187194us) write_op.cc:301] APPLY: starting
0709 13:06:13.537751 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:13.752536 (+214785us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:13.752696 (+   160us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:14.257836 (+505140us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:14.258248 (+   412us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 13:06:14.258296 (+    48us) write_op.cc:312] APPLY: finished
0709 13:06:14.258881 (+   585us) log.cc:844] Serialized 220 byte log entry
0709 13:06:14.488751 (+229870us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:14.488972 (+   221us) write_op.cc:454] Released schema lock
0709 13:06:14.489061 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:14.493186 (+  4125us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:14.493221 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5180697,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":186869,"apply.run_cpu_time_us":37740,"apply.run_wall_time_us":955500,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"num_ops":25,"prepare.queue_time_us":229435,"prepare.run_cpu_time_us":35113,"prepare.run_wall_time_us":419509,"raft.queue_time_us":290753,"raft.run_cpu_time_us":5200,"raft.run_wall_time_us":33638,"replication_time_us":709998,"thread_start_us":685701,"threads_started":5,"wal-append.queue_time_us":404249,"wal-append.run_cpu_time_us":7582,"wal-append.run_wall_time_us":1219113}]]}
I20260709 13:06:15.101603  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:15.105307 15080 leader_election.cc:290] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:15.128258 15594 consensus_queue.cc:1059] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:15.129405 15594 raft_consensus.cc:3055] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:15.129914 15594 raft_consensus.cc:740] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:15.130774 15594 consensus_queue.cc:260] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:15.135573  9086 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.142580  9194 consensus_queue.cc:237] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.170367  9269 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 14. Current term is 15. Ops: []
I20260709 13:06:15.143803 15594 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:15.151799  9120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:15.151927  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8f97d26e68b4fbead130c830dbf4b51" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:15.169145  9198 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:15.190302 15563 consensus_queue.cc:1059] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:15.197846  9855 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
W20260709 13:06:15.222345  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 290802us
W20260709 13:06:15.225072  9162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:06:15.227017  9853 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7597d602dc674623ae1d5de5fdb5d2ee" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:15.199457 15270 consensus_queue.cc:1059] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:06:15.246066 15270 consensus_queue.cc:1243] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Peer 94504d29c6744f38bb06943479df9466 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:06:15.246644 15270 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:15.246980 15270 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:15.252444 15563 consensus_queue.cc:1243] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer 1efabcb1befc400aadf101e8276fc1aa log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:06:15.254493 15563 raft_consensus.cc:3055] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:15.271064  9269 raft_consensus.cc:1240] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Rejecting Update request from peer 92919b70789041c8a0de0b3525154fd1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:06:15.254876 15563 raft_consensus.cc:740] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:06:15.317834  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 47737.2us
I20260709 13:06:15.321907 15270 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.323421 15270 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:15.325325 15563 consensus_queue.cc:260] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:15.329809  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 54721.2us
I20260709 13:06:15.346271  9118 raft_consensus.cc:1240] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 14. Current term is 15. Ops: []
W20260709 13:06:15.354492 15644 log.cc:927] Time spent T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:06:15.374729  9269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2655d16bd349309dab52831e66fee8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:15.379130  9164 leader_election.cc:304] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:15.361547 15563 raft_consensus.cc:3060] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:15.378546 15530 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:06:15.422278  9198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:15.425657  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:15.427975  9087 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:15.422921  9198 raft_consensus.cc:606] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:15.442994  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:15.448349  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:15.458531 15511 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:15.461732  9238 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:15.462994 15289 raft_consensus.cc:2749] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:15.468526  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 47249.8us
I20260709 13:06:15.469650 14809 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:15.472607  9118 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:15.473382  9264 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:06:15.476697 15523 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:06:15.400043 15528 raft_consensus.cc:2749] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:15.513950  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 134899us
I20260709 13:06:15.445323 15548 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:06:15.501142 15522 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:06:15.567035  9197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a06199955d24d13ba9589faa3f11ebe"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:15.567943  9197 raft_consensus.cc:606] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:15.568440  9197 raft_consensus.cc:614] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:06:15.574831  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 105609us
W20260709 13:06:15.581534  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 119887us
I20260709 13:06:15.594302  9857 raft_consensus.cc:1275] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:06:15.595974  9197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:15.596613  9197 raft_consensus.cc:606] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:15.607692  9197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:15.614009  9197 raft_consensus.cc:606] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:15.614492  9197 raft_consensus.cc:614] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:15.664582 15439 raft_consensus.cc:3055] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:15.665230 15439 raft_consensus.cc:740] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:15.666285 15439 consensus_queue.cc:260] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.642009 15523 consensus_queue.cc:1048] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:06:15.697151  9862 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:06:15.697623  9862 pending_rounds.cc:85] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Aborting all ops after (but not including) 14
I20260709 13:06:15.697851  9862 pending_rounds.cc:107] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:06:15.698088  9862 raft_consensus.cc:2889] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:15.736510 15522 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:15.747064 15529 raft_consensus.cc:993] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:06:15.747831 15529 raft_consensus.cc:1081] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:06:15.750425  9859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:06:15.751143  9859 raft_consensus.cc:493] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:15.751556  9859 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:15.761045  9198 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:06:15.761919  9198 pending_rounds.cc:85] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Aborting all ops after (but not including) 9
I20260709 13:06:15.762458  9198 pending_rounds.cc:107] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:06:15.762794  9198 raft_consensus.cc:2889] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:15.763513 15594 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.764081 15594 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:15.766328 15594 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:15.768714  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:15.768924  9859 raft_consensus.cc:515] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.771226  9862 raft_consensus.cc:1240] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Rejecting Update request from peer 94504d29c6744f38bb06943479df9466 for earlier term 13. Current term is 14. Ops: []
I20260709 13:06:15.774039 15522 consensus_queue.cc:1059] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:15.782903 15439 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:15.787634  9198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:15.788462  9198 raft_consensus.cc:606] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:15.789796 15357 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:15.799115 15270 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.801805  9859 leader_election.cc:290] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:15.803535  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:15.799698 15270 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.804351  9122 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:15.807523  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:15.808522  9270 raft_consensus.cc:2410] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:06:15.811138 15270 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:15.813092  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:15.813657  9197 raft_consensus.cc:3055] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:06:15.813920  9197 raft_consensus.cc:740] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:15.814539  9197 consensus_queue.cc:260] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.816063  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:15.816857  9190 raft_consensus.cc:2410] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:06:15.818403  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:15.815634  9197 raft_consensus.cc:3060] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:15.820325  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:15.825323 15518 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:15.825940 15518 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.827116  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:15.827688  9194 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:06:15.834081  9197 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
W20260709 13:06:15.835839  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 214325us
I20260709 13:06:15.835986  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:15.837774  9118 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 16.
I20260709 13:06:15.840183 15518 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:15.841187  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:15.842893  9118 raft_consensus.cc:1275] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:06:15.848045  9825 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:15.850080 15357 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:15.856150 15557 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:15.857511  9122 raft_consensus.cc:2468] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
W20260709 13:06:15.891469  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 43524.3us
I20260709 13:06:15.863903  9089 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:15.879427  9828 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:15.861564 15436 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Committing config change with OpId 10.10: config changed from index 7 to 10, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:15.893241  9825 leader_election.cc:304] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:15.897044 15505 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:15.906294 15568 raft_consensus.cc:2804] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:15.906941 15568 raft_consensus.cc:697] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:15.909299 15568 consensus_queue.cc:237] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.897518 15511 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:15.912161 15511 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.912685 15511 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:15.898264 15557 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:15.894258 15357 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.897886 15594 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:15.921213 15594 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:15.921555 15594 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:15.920083  9118 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Committing config change with OpId 10.10: config changed from index 7 to 10, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:15.929857 15605 consensus_queue.cc:1048] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:06:15.927774 15640 raft_consensus.cc:493] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:15.939121 15640 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:15.941365 15640 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:15.944041  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:15.949220 15357 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:15.956760  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:15.955803  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:15.929373  9860 raft_consensus.cc:2955] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Committing config change with OpId 10.10: config changed from index 7 to 10, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:15.960244  9116 raft_consensus.cc:2393] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 16: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:15.957407  9194 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
I20260709 13:06:15.962886  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:15.966109  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:15.967146  9857 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:15.976194 15502 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:15.987705 15502 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:15.988161 15502 raft_consensus.cc:3060] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:15.994084 15557 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:15.996372  9194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:16.002094  9194 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 12: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:16.011574  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:15.994899 15436 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:39054 (request call id 3) took 1028 ms. Trace:
I20260709 13:06:16.033746 15594 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:16.035174  9116 raft_consensus.cc:2368] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13.
I20260709 13:06:16.017910 15436 rpcz_store.cc:276] 0709 13:06:14.966973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:14.967351 (+   378us) service_pool.cc:224] Handling call
0709 13:06:15.141807 (+174456us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b58007928d0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566400aa 00007f76566c2447 00007f76566df70a 00007f76566e076a 00007f76566df91b 00007f76566eb03c 00007f76566ea6b3 00007f76566e29fa 00007f765b6aa4c9 00007f7654c26c54
0709 13:06:15.172646 (+ 30839us) log.cc:844] Serialized 494 byte log entry
0709 13:06:15.317851 (+145205us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b2c000fc1b8. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f7654fdda11 00007f7654feca0a 00007f7654fec693 00007f7654fec631 00007f7656720b52 00007f7656720cf5 00007f765665d691 00007f765663deaa 00007f7656644d53 00007f7656644cd9 00007f7656644c69
0709 13:06:15.456062 (+138211us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b18000deae0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f7656720d01 00007f765665d691 00007f765663deaa 00007f7656644d53 00007f7656644cd9 00007f7656644c69 00007f7656644c31 00007f7656643dfd 000000000038f354 000000000038f199
0709 13:06:15.994808 (+538746us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55498,"raft.queue_time_us":55928,"raft.run_cpu_time_us":4947,"raft.run_wall_time_us":214153,"spinlock_wait_cycles":442159488,"thread_start_us":47825,"threads_started":1,"wal-append.queue_time_us":169721}
I20260709 13:06:16.044721  9008 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1c199cae5ff14a478726c8a8b491f8f7 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:16.047425 15511 raft_consensus.cc:515] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.059731 15594 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:16.064728  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:16.065034  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:16.065428  9190 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:16.065614  9264 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:06:16.066897  9859 raft_consensus.cc:2368] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:16.067497 15502 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.071302 15502 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:16.071416  9235 leader_election.cc:400] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13.
I20260709 13:06:16.071780  9235 leader_election.cc:403] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:16.081427  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:16.081491  9238 leader_election.cc:400] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:16.082099  9238 leader_election.cc:403] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:06:16.083262  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:16.084154  9859 raft_consensus.cc:2393] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 17: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:16.084669 15443 raft_consensus.cc:993] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:16.085137 15443 raft_consensus.cc:1081] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:16.088332  9266 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:06:16.088928  9266 raft_consensus.cc:493] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:16.088889  9264 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:06:16.089447  9266 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:16.090931 14986 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939) with higher term. Message: Invalid argument: T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 12. Current term is 13.
I20260709 13:06:16.094231  9087 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:16.096684 15349 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:16.097255 15349 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:16.097859 15511 leader_election.cc:290] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:16.089946 15670 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:06:16.098117 15349 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:16.101034  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:16.101648  9116 raft_consensus.cc:3060] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:16.108351  9198 raft_consensus.cc:2468] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
I20260709 13:06:16.109599  9116 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:16.110831  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd9140502bd3469eb208753dea0f3159" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:16.112720  9825 leader_election.cc:304] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:16.128322  9238 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:16.132031 15511 raft_consensus.cc:2804] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:16.132943 15511 raft_consensus.cc:697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:16.134241  9190 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 13.
I20260709 13:06:16.134167 15511 consensus_queue.cc:237] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.147549  9266 raft_consensus.cc:515] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:16.150734  9266 leader_election.cc:290] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:16.151577  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:16.152202  9116 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:16.152918  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90c9bcfd54594ec58c5e8e1be6c556d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:16.153548  9190 raft_consensus.cc:3055] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:06:16.153841  9190 raft_consensus.cc:740] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:16.151407 15640 raft_consensus.cc:2804] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:16.160646  9190 consensus_queue.cc:260] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:16.166239  9190 raft_consensus.cc:3060] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:16.170715  9116 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:16.171922  9235 leader_election.cc:304] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:16.173003 15562 raft_consensus.cc:2804] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:16.173535 15562 raft_consensus.cc:697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:16.180037  9190 raft_consensus.cc:2468] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:16.169451 15640 raft_consensus.cc:697] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:16.174598 15562 consensus_queue.cc:237] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:16.204345 15670 raft_consensus.cc:2749] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523) with higher term. Message: Invalid argument: T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa for earlier term 16. Current term is 17.
I20260709 13:06:16.228673  9264 raft_consensus.cc:2468] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:16.231181 15555 raft_consensus.cc:993] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:06:16.231678 15555 raft_consensus.cc:1081] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:06:16.239439  9859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:06:16.240149  9859 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:16.240679  9859 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:16.224573 15640 consensus_queue.cc:237] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.289883 15605 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:16.290603 15605 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.293733 15605 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:16.296875  9118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:16.334091  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:16.341172  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:16.336138  9859 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.343611  9859 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:16.352192  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:16.352207  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:16.352851  9190 raft_consensus.cc:3055] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:16.353096  9116 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:16.353205  9190 raft_consensus.cc:740] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:16.354004  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:16.354262  9190 consensus_queue.cc:260] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.360110 15640 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:16.386770  9116 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:16.388774  9825 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:16.395601 15104 raft_consensus.cc:2804] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:16.412010 15104 raft_consensus.cc:697] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:16.430159 15104 consensus_queue.cc:237] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.435464  9190 raft_consensus.cc:3060] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:16.473301  9190 raft_consensus.cc:2468] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:16.509759  9198 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:16.511173 15411 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.511698 15411 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.514101 15411 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:16.515801  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:16.516387  9198 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
I20260709 13:06:16.517412 15411 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:16.518607  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:16.529208  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:16.543320 15511 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:16.546613 15511 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.546979 15511 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:16.558419  9116 raft_consensus.cc:1275] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:16.590193 15104 consensus_queue.cc:1048] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:06:16.621165 15104 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.621891 15104 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:16.628347  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 76368.5us
I20260709 13:06:16.816493 15511 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.819240 15511 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:16.821360  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:16.822005  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 51383.5us
I20260709 13:06:16.827240 15104 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:16.867514  9014 catalog_manager.cc:5697] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 18 to 19, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 19 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:16.875630 15594 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:16.877068  9116 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:16.880385  9198 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:16.884831  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:16.890379  9189 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:16.902338  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:16.902788  9189 raft_consensus.cc:2468] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
I20260709 13:06:16.903337  9197 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 19.
I20260709 13:06:16.906047  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:16.908576  9190 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:16.916127  9828 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:16.919247  9190 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:06:16.917668  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 43650.8us
I20260709 13:06:16.917932  9270 raft_consensus.cc:1275] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:16.920552 15683 raft_consensus.cc:2804] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:16.921131 15683 raft_consensus.cc:697] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:16.920746  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:16.927100 15568 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:06:16.927516 15568 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:16.927886 15568 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:16.931187  9116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:16.936452  9014 catalog_manager.cc:5697] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:16.938156 15270 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:16.922080 15683 consensus_queue.cc:237] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.938184  9014 catalog_manager.cc:5697] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:16.947911 15349 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:06:16.957728  9014 catalog_manager.cc:5697] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 13 to 15, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 15 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:16.959287  9118 raft_consensus.cc:1275] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:06:16.959198  9014 catalog_manager.cc:5697] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 18, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 18 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:16.961565  9014 catalog_manager.cc:5697] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:16.967931  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 44737.4us
I20260709 13:06:16.963475  9014 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:16.972867  9014 catalog_manager.cc:5697] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 5 to 11, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 11 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:16.898160 15594 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:16.986516  9855 raft_consensus.cc:1275] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:06:16.990567  9198 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:17.010046  9014 catalog_manager.cc:5697] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 12, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 12 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.011605  9014 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 6 to 13, 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.013146  9014 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 17 to 20, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 20 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.026769  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:17.027611  9189 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 19.
W20260709 13:06:17.007728  9021 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42488 (request call id 242) took 43144 ms (client timeout 15000 ms). Trace:
I20260709 13:06:17.015136 15568 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:17.021799  9116 raft_consensus.cc:2410] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:06:17.007352 15594 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:17.025161  9014 catalog_manager.cc:5697] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.035002  9014 catalog_manager.cc:5697] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 8 to 18. New cstate: current_term: 18 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.039495  9014 catalog_manager.cc:5697] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 12 to 13, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 13 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:17.041270  9014 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.047844  9117 raft_consensus.cc:1275] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:17.033656  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:17.043066  9014 catalog_manager.cc:5697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.052814  9014 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 14 to 18, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 18 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.061513 15499 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:06:17.067729  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:17.071341  9859 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 20: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:17.066557  9014 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:17.028692  9021 rpcz_store.cc:269] 0709 13:05:33.863913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:33.864242 (+   329us) service_pool.cc:224] Handling call
0709 13:06:17.007645 (+43143403us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:15.239531 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:15.240118 (+   587us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:15.240155 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:06:15.240173 (+    18us) tablet.cc:662] Decoding operations
0709 13:06:15.384468 (+144295us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:15.384568 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:15.384598 (+    30us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:06:15.486021 (+101423us) tablet.cc:701] Row locks acquired
0709 13:06:15.486055 (+    34us) write_op.cc:260] PREPARE: finished
0709 13:06:15.486304 (+   249us) write_op.cc:270] Start()
0709 13:06:15.486494 (+   190us) write_op.cc:276] Timestamp: P: 1783602375486227 usec, L: 0
0709 13:06:15.486528 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:06:15.487474 (+   946us) log.cc:844] Serialized 4334 byte log entry
0709 13:06:15.994053 (+506579us) op_driver.cc:464] REPLICATION: finished
0709 13:06:16.150073 (+156020us) write_op.cc:301] APPLY: starting
0709 13:06:16.151634 (+  1561us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:16.304596 (+152962us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:16.304693 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:16.408629 (+103936us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:16.425818 (+ 17189us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 13:06:16.425899 (+    81us) write_op.cc:312] APPLY: finished
0709 13:06:16.426367 (+   468us) log.cc:844] Serialized 105 byte log entry
0709 13:06:16.426611 (+   244us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:16.426808 (+   197us) write_op.cc:454] Released schema lock
0709 13:06:16.426876 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:16.427420 (+   544us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:16.427463 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2740785,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":155165,"apply.run_cpu_time_us":17596,"apply.run_wall_time_us":277317,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50129,"num_ops":11,"prepare.queue_time_us":164957,"prepare.run_cpu_time_us":44010,"prepare.run_wall_time_us":437642,"raft.queue_time_us":141965,"raft.run_cpu_time_us":14934,"raft.run_wall_time_us":70827,"replication_time_us":507343,"thread_start_us":514680,"threads_started":4,"wal-append.queue_time_us":355194}]]}
I20260709 13:06:17.079949  9014 catalog_manager.cc:5697] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 18, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 18 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.085454  9014 catalog_manager.cc:5697] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:17.086690  9014 catalog_manager.cc:5697] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 12 to 15, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 15 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.087956  9014 catalog_manager.cc:5697] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 15 to 19, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 19 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.089226  9014 catalog_manager.cc:5697] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 13 to 14, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 14 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.090461  9014 catalog_manager.cc:5697] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.091610  9014 catalog_manager.cc:5697] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 9 to 14, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 14 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.056289 15357 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:06:17.110759  9014 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.065040 15270 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:06:17.113862  9014 catalog_manager.cc:5697] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 17, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 17 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.074909 15686 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:06:17.114769  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 80565.1us
I20260709 13:06:17.128754  9014 catalog_manager.cc:5697] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 16, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 16 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
W20260709 13:06:17.129499  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 48697us
I20260709 13:06:17.130201  9014 catalog_manager.cc:5697] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 15 to 16, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 16 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:17.132021  9014 catalog_manager.cc:5697] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.133852  9014 catalog_manager.cc:5697] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 14 to 17, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 17 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.135586  9014 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 8 to 12, term changed from 14 to 15, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to <none>, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 15 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.137585  9014 catalog_manager.cc:5697] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 17, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 17 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.139492  9014 catalog_manager.cc:5697] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:17.141367  9014 catalog_manager.cc:5697] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.133354 15686 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:17.132474 15289 consensus_queue.cc:1048] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:06:17.173759 15696 consensus_queue.cc:1048] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:17.164546 15568 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:17.152249 15502 consensus_queue.cc:1048] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:06:17.156515 15686 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:17.178661  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 84522.8us
I20260709 13:06:17.185710  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
W20260709 13:06:17.200374 15689 log.cc:927] Time spent T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: Append to log took a long time: real 0.051s	user 0.003s	sys 0.000s
W20260709 13:06:17.207979  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 124030us
I20260709 13:06:17.252259 15499 raft_consensus.cc:493] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:17.252861 15499 raft_consensus.cc:515] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:17.270340 15499 leader_election.cc:290] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:17.290555 15686 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:17.291900  9122 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 20: Already voted for candidate ce51b61da22f40d4aa0204a6b1028f28 in this term.
I20260709 13:06:17.297241 15686 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:17.297780  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:17.298507  9855 raft_consensus.cc:2393] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ce51b61da22f40d4aa0204a6b1028f28 in current term 20: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:17.318332  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:17.318938  9190 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:06:17.328943  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 326901us
I20260709 13:06:17.337633  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
W20260709 13:06:17.340420  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 123796us
I20260709 13:06:17.377127  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:17.376972 15511 consensus_queue.cc:1048] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:17.378630 15104 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:06:17.380630  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 163445us
I20260709 13:06:17.405354  9198 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:17.407362 15104 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:17.426609  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:17.416339  9190 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 20.
I20260709 13:06:17.461323  9089 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:17.463413 15686 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:17.464298 15686 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:17.465649  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cfb4241815c471a8b29031ba9674528" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:17.466671 15686 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:17.537516  9264 raft_consensus.cc:3060] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:06:17.543859  9266 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:06:17.548183  9264 raft_consensus.cc:1275] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:06:17.579936  9238 leader_election.cc:304] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:17.581880 15707 raft_consensus.cc:2749] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:17.603637 15686 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:06:17.607075 15104 consensus_queue.cc:1048] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:06:17.683051  9016 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: config changed from index 12 to 8, term changed from 15 to 12, leader changed from <none> to 94504d29c6744f38bb06943479df9466 (127.7.213.66), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) added. New cstate: current_term: 12 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.701206  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 017b65b967c44e7688e66e7aab235b62 with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:45222
I20260709 13:06:17.708501  9014 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36626 (request call id 363) took 1522 ms. Trace:
I20260709 13:06:17.719190 15726 tablet_replica.cc:333] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:17.720675 15726 raft_consensus.cc:2243] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:17.727713 15726 raft_consensus.cc:2272] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:17.731652 15726 ts_tablet_manager.cc:1916] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:17.710942  9014 rpcz_store.cc:276] 0709 13:06:16.186137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.187513 (+  1376us) service_pool.cc:224] Handling call
0709 13:06:17.708406 (+1520893us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:17.240443 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:17.258962 (+ 18519us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:17.260790 (+  1828us) write_op.cc:435] Acquired schema lock
0709 13:06:17.268115 (+  7325us) tablet.cc:662] Decoding operations
0709 13:06:17.278795 (+ 10680us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:17.278893 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:17.278934 (+    41us) tablet.cc:685] Acquiring locks for 35 operations
0709 13:06:17.286427 (+  7493us) tablet.cc:701] Row locks acquired
0709 13:06:17.286466 (+    39us) write_op.cc:260] PREPARE: finished
0709 13:06:17.286661 (+   195us) write_op.cc:270] Start()
0709 13:06:17.286802 (+   141us) write_op.cc:276] Timestamp: P: 1783602377286604 usec, L: 0
0709 13:06:17.286835 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:06:17.287683 (+   848us) log.cc:844] Serialized 13595 byte log entry
0709 13:06:17.374257 (+ 86574us) op_driver.cc:464] REPLICATION: finished
0709 13:06:17.405817 (+ 31560us) write_op.cc:301] APPLY: starting
0709 13:06:17.405929 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:17.434529 (+ 28600us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b1800194bc0. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f765628c4bc 00007f76535da48b 00007f765c434ab9 00007f76535f669b 00007f76535f493f 00007f76535f6c02 00007f765a11aef0 00007f765a11b489 00007f765a11d71a
0709 13:06:17.537283 (+102754us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:17.537312 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:17.639233 (+101921us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:17.639650 (+   417us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=70,key_file_lookups=70,delta_file_lookups=0,mrs_lookups=0
0709 13:06:17.639690 (+    40us) write_op.cc:312] APPLY: finished
0709 13:06:17.640196 (+   506us) log.cc:844] Serialized 300 byte log entry
0709 13:06:17.642313 (+  2117us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:17.645653 (+  3340us) write_op.cc:454] Released schema lock
0709 13:06:17.645729 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:17.653547 (+  7818us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:17.653577 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":100427,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":31289,"apply.run_cpu_time_us":41943,"apply.run_wall_time_us":243664,"cfile_cache_hit":142,"cfile_cache_hit_bytes":150497,"num_ops":35,"prepare.queue_time_us":72011,"prepare.run_cpu_time_us":8665,"prepare.run_wall_time_us":67648,"replication_time_us":87255,"spinlock_wait_cycles":52235648,"thread_start_us":106626,"threads_started":2}]]}
I20260709 13:06:17.759548 15726 ts_tablet_manager.cc:1929] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.11
I20260709 13:06:17.760195 15726 log.cc:1199] T 017b65b967c44e7688e66e7aab235b62 P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/017b65b967c44e7688e66e7aab235b62
I20260709 13:06:17.804939  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 017b65b967c44e7688e66e7aab235b62 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:06:17.809401  9017 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.812004  9017 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 9 to 11, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 11 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:17.814246  9017 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 10 to 11, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 11 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:17.821818  9016 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36610 (request call id 327) took 55143 ms (client timeout 15000 ms). Trace:
W20260709 13:06:17.822744  9016 rpcz_store.cc:269] 0709 13:05:22.678483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:22.678854 (+   371us) service_pool.cc:224] Handling call
0709 13:06:17.821730 (+55142876us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:17.704458 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:17.705173 (+   715us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:17.705222 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:06:17.705247 (+    25us) tablet.cc:662] Decoding operations
0709 13:06:17.705674 (+   427us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:17.705744 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:17.705787 (+    43us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:17.706022 (+   235us) tablet.cc:701] Row locks acquired
0709 13:06:17.706046 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:06:17.706217 (+   171us) write_op.cc:270] Start()
0709 13:06:17.706355 (+   138us) write_op.cc:276] Timestamp: P: 1783602377706163 usec, L: 0
0709 13:06:17.706383 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:06:17.707043 (+   660us) log.cc:844] Serialized 595 byte log entry
0709 13:06:17.709496 (+  2453us) op_driver.cc:464] REPLICATION: finished
0709 13:06:17.709824 (+   328us) write_op.cc:301] APPLY: starting
0709 13:06:17.709907 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:17.710646 (+   739us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:17.710737 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:17.711361 (+   624us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:17.711516 (+   155us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:06:17.711559 (+    43us) write_op.cc:312] APPLY: finished
0709 13:06:17.711724 (+   165us) log.cc:844] Serialized 25 byte log entry
0709 13:06:17.711888 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:17.711963 (+    75us) write_op.cc:454] Released schema lock
0709 13:06:17.712000 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:17.712346 (+   346us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:17.712377 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1085935,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":104,"apply.run_cpu_time_us":2478,"apply.run_wall_time_us":2478,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":128,"prepare.run_cpu_time_us":2895,"prepare.run_wall_time_us":2895,"replication_time_us":2965}]]}
I20260709 13:06:17.826700  9017 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.829071  9017 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 9 to 10, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 10 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:17.830432  9017 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 7 to 10, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 10 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.832594  9017 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:17.834503  9017 catalog_manager.cc:5697] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:18.005065 15501 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:18.006132 15501 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } attrs { replace: true } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:18.009217 15501 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:18.028122  9015 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.032910  9015 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.037586  9015 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:06:18.039386  9017 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36610 (request call id 328) took 32027 ms (client timeout 15000 ms). Trace:
W20260709 13:06:18.042479  9017 rpcz_store.cc:269] 0709 13:05:46.017072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:46.023403 (+  6331us) service_pool.cc:224] Handling call
0709 13:06:18.039294 (+32015891us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:17.892460 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:17.893058 (+   598us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:17.893094 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:06:17.893116 (+    22us) tablet.cc:662] Decoding operations
0709 13:06:17.894040 (+   924us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:17.894097 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:17.894126 (+    29us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:06:17.894835 (+   709us) tablet.cc:701] Row locks acquired
0709 13:06:17.894853 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:06:17.894975 (+   122us) write_op.cc:270] Start()
0709 13:06:17.895086 (+   111us) write_op.cc:276] Timestamp: P: 1783602377894936 usec, L: 0
0709 13:06:17.895107 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:06:17.895737 (+   630us) log.cc:844] Serialized 3190 byte log entry
0709 13:06:17.898314 (+  2577us) op_driver.cc:464] REPLICATION: finished
0709 13:06:17.898659 (+   345us) write_op.cc:301] APPLY: starting
0709 13:06:17.898748 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:17.902977 (+  4229us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:17.903078 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:17.907737 (+  4659us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:17.909061 (+  1324us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:06:17.909116 (+    55us) write_op.cc:312] APPLY: finished
0709 13:06:17.909401 (+   285us) log.cc:844] Serialized 81 byte log entry
0709 13:06:17.910233 (+   832us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:17.910349 (+   116us) write_op.cc:454] Released schema lock
0709 13:06:17.910403 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:17.952052 (+ 41649us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:17.952101 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":49296,"reported_tablets":45,"spinlock_wait_cycles":33152,"child_traces":[["op",{"apply.queue_time_us":116,"apply.run_cpu_time_us":10422,"apply.run_wall_time_us":53313,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33456,"num_ops":8,"prepare.queue_time_us":109,"prepare.run_cpu_time_us":3597,"prepare.run_wall_time_us":5714,"replication_time_us":3092}]]}
I20260709 13:06:18.057933  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:18.059154  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:18.059324  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28" is_pre_election: true
I20260709 13:06:18.064853  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:18.069233  9197 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:18.049496  9015 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.093883  9857 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:18.095338 15686 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:18.107671  9015 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.111972 15654 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:18.117735  9015 catalog_manager.cc:5697] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.128486  9015 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.133810  9015 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 10 to 11, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 11 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:18.128505  9238 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:18.135854  9015 catalog_manager.cc:5697] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.138914 15707 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:18.237354  9014 catalog_manager.cc:5697] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 18 to 19, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 19 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.238778  9014 catalog_manager.cc:5697] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 8 to 12, term changed from 12 to 15, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68), VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 15 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.240302  9014 catalog_manager.cc:5697] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.254583  9014 catalog_manager.cc:5697] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 7 to 17, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 17 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.256358  9014 catalog_manager.cc:5697] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: config changed from index 7 to 10, VOTER 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) evicted. New cstate: current_term: 10 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:18.258538  9014 catalog_manager.cc:5697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 10 to 13, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 13 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:18.262184  9014 catalog_manager.cc:5697] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 13 to 14, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 14 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
W20260709 13:06:18.270516  9015 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36634 (request call id 343) took 16977 ms (client timeout 15000 ms). Trace:
W20260709 13:06:18.270937  9015 rpcz_store.cc:269] 0709 13:06:01.293273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:01.295671 (+  2398us) service_pool.cc:224] Handling call
0709 13:06:01.542078 (+246407us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b540001a4d4. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664e090 00007f76566ecf3a 00007f765c4a31cc 00007f765c5a137e 00007f765aa622a4 00007f765aa62232 00007f765aa62161 00007f765aa620dc 00007f765aa61392 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:18.270449 (+16728371us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:18.148073 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:18.148662 (+   589us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:18.148705 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:06:18.148722 (+    17us) tablet.cc:662] Decoding operations
0709 13:06:18.149691 (+   969us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:18.149750 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:18.149782 (+    32us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:06:18.150567 (+   785us) tablet.cc:701] Row locks acquired
0709 13:06:18.150589 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:06:18.150740 (+   151us) write_op.cc:270] Start()
0709 13:06:18.153496 (+  2756us) write_op.cc:276] Timestamp: P: 1783602378150696 usec, L: 0
0709 13:06:18.153540 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:06:18.154223 (+   683us) log.cc:844] Serialized 3570 byte log entry
0709 13:06:18.162064 (+  7841us) op_driver.cc:464] REPLICATION: finished
0709 13:06:18.162921 (+   857us) write_op.cc:301] APPLY: starting
0709 13:06:18.163048 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:18.169101 (+  6053us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:18.169219 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:18.194898 (+ 25679us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:18.195209 (+   311us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:06:18.195260 (+    51us) write_op.cc:312] APPLY: finished
0709 13:06:18.195539 (+   279us) log.cc:844] Serialized 89 byte log entry
0709 13:06:18.199980 (+  4441us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:18.200161 (+   181us) write_op.cc:454] Released schema lock
0709 13:06:18.200227 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:18.200649 (+   422us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:18.200676 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":35649,"reported_tablets":46,"spinlock_wait_cycles":107579648,"child_traces":[["op",{"apply.queue_time_us":436,"apply.run_cpu_time_us":14486,"apply.run_wall_time_us":37730,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37638,"num_ops":9,"prepare.queue_time_us":4237,"prepare.run_cpu_time_us":4006,"prepare.run_wall_time_us":13715,"replication_time_us":8304,"spinlock_wait_cycles":1859456}]]}
W20260709 13:06:18.293596  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 48189.8us
I20260709 13:06:18.408962  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "017b65b967c44e7688e66e7aab235b62"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:18.409744  9190 raft_consensus.cc:606] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:18.410135  9190 raft_consensus.cc:614] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:18.439671  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:18.440444  9190 raft_consensus.cc:606] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:18.444226  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:18.444917  9190 raft_consensus.cc:606] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Received request to transfer leadership to TS 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:18.464663  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c089688bfc69403095596a002ccbf890"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:18.465314  9190 raft_consensus.cc:606] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:18.477021  9190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f84c9fc901364e60b2c414ee3b7a7e65"
dest_uuid: "94504d29c6744f38bb06943479df9466"
mode: GRACEFUL
new_leader_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:46626
I20260709 13:06:18.477677  9190 raft_consensus.cc:606] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Received request to transfer leadership to TS 1efabcb1befc400aadf101e8276fc1aa
I20260709 13:06:18.478027  9190 raft_consensus.cc:614] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:06:18.481583  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:18.483480  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:18.484820 15529 raft_consensus.cc:993] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: : Instructing follower 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:06:18.485373 15529 raft_consensus.cc:1081] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Signalling peer 92919b70789041c8a0de0b3525154fd1 to start an election
I20260709 13:06:18.487280  9857 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e"
dest_uuid: "92919b70789041c8a0de0b3525154fd1"
 from {username='slave'} at 127.0.0.1:40108
I20260709 13:06:18.487823  9857 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.488166  9857 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:18.511994  9857 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:18.516376  9857 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:18.519351  9115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:18.520045  9115 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:18.529043  9115 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:18.530423  9825 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:18.533649 15104 raft_consensus.cc:2804] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:18.537602 15104 raft_consensus.cc:697] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:18.538539 15104 consensus_queue.cc:237] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:18.577194  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 017b65b967c44e7688e66e7aab235b62 with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:45222
I20260709 13:06:18.578830  9259 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c199cae5ff14a478726c8a8b491f8f7 with delete_type TABLET_DATA_TOMBSTONED (TS 1efabcb1befc400aadf101e8276fc1aa not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45222
W20260709 13:06:18.592339  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 136133us
I20260709 13:06:18.596060  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:18.596704  9189 raft_consensus.cc:3055] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:18.597007  9189 raft_consensus.cc:740] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:18.597765  9189 consensus_queue.cc:260] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:18.599071  9189 raft_consensus.cc:3060] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:18.619447  9189 raft_consensus.cc:2468] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:18.651798  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 017b65b967c44e7688e66e7aab235b62 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:06:18.653756 15740 tablet_replica.cc:333] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:18.664032 15697 raft_consensus.cc:993] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:18.664666 15736 raft_consensus.cc:1081] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:18.666568 15740 raft_consensus.cc:2243] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:18.667315 15740 raft_consensus.cc:2272] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:18.670426 15740 ts_tablet_manager.cc:1916] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:18.659713  9014 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:18.698523  9264 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c089688bfc69403095596a002ccbf890"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:06:18.699235  9264 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.699672  9264 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.717298 15687 raft_consensus.cc:993] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: : Instructing follower 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:18.717870 15687 raft_consensus.cc:1081] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Signalling peer 1efabcb1befc400aadf101e8276fc1aa to start an election
I20260709 13:06:18.740563  9020 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 8 to 10, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 10 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:18.758424  9267 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
 from {username='slave'} at 127.0.0.1:39752
I20260709 13:06:18.759189  9267 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:18.759613  9267 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:18.820334 15740 ts_tablet_manager.cc:1929] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 10.9
I20260709 13:06:18.820915 15740 log.cc:1199] T 1c199cae5ff14a478726c8a8b491f8f7 P 1efabcb1befc400aadf101e8276fc1aa: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/wals/1c199cae5ff14a478726c8a8b491f8f7
I20260709 13:06:18.822781  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 1c199cae5ff14a478726c8a8b491f8f7 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:06:18.813853  9267 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:18.836361  9267 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:18.837491  9264 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:18.839087  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:18.849853  9264 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:18.850719  9122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
I20260709 13:06:18.859880  9122 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.851172  9190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:18.864075  9190 raft_consensus.cc:3055] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:18.864411  9190 raft_consensus.cc:740] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:18.865123  9190 consensus_queue.cc:260] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:18.866122  9190 raft_consensus.cc:3060] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:18.850395  9189 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:06:18.883987  9189 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:18.886523  9189 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:18.887822  9189 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:18.890473  9122 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:06:18.891611  9235 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:18.892474 15289 raft_consensus.cc:2804] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:18.892982 15289 raft_consensus.cc:697] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:18.893850 15289 consensus_queue.cc:237] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:18.903023  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:18.903646  9859 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:18.917822  9859 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 19.
I20260709 13:06:18.885560  9190 raft_consensus.cc:2468] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:06:18.953817  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c199cae5ff14a478726c8a8b491f8f7 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 10)) from {username='slave'} at 127.0.0.1:45222
I20260709 13:06:18.955583  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 1c199cae5ff14a478726c8a8b491f8f7 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:06:18.959714  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:18.972386  9189 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 19.
I20260709 13:06:18.973594  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 017b65b967c44e7688e66e7aab235b62 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 12)) from {username='slave'} at 127.0.0.1:45222
I20260709 13:06:18.970068 15289 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:18.978083 15289 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:18.983784 15289 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:18.980722  9020 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36634 (request call id 342) took 39268 ms (client timeout 15000 ms). Trace:
I20260709 13:06:18.978725  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 017b65b967c44e7688e66e7aab235b62 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
W20260709 13:06:18.988385  9020 rpcz_store.cc:269] 0709 13:05:39.712804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:39.789160 (+ 76356us) service_pool.cc:224] Handling call
0709 13:06:18.756594 (+38967434us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:18.756619 (+    25us) write_op.cc:454] Released schema lock
0709 13:06:18.980652 (+224033us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:18.749184 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:18.749726 (+   542us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:18.749777 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:06:18.749789 (+    12us) tablet.cc:662] Decoding operations
0709 13:06:18.750124 (+   335us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:18.750179 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:18.750206 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:18.750380 (+   174us) tablet.cc:701] Row locks acquired
0709 13:06:18.750396 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:06:18.750535 (+   139us) write_op.cc:270] Start()
0709 13:06:18.750640 (+   105us) write_op.cc:276] Timestamp: P: 1783602378750494 usec, L: 0
0709 13:06:18.750655 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:06:18.751218 (+   563us) log.cc:844] Serialized 533 byte log entry
0709 13:06:18.753101 (+  1883us) op_driver.cc:464] REPLICATION: finished
0709 13:06:18.753503 (+   402us) write_op.cc:301] APPLY: starting
0709 13:06:18.753581 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:18.754291 (+   710us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:18.754381 (+    90us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:18.755026 (+   645us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:18.755166 (+   140us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:06:18.755210 (+    44us) write_op.cc:312] APPLY: finished
0709 13:06:18.755377 (+   167us) log.cc:844] Serialized 25 byte log entry
0709 13:06:18.755527 (+   150us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:18.755604 (+    77us) write_op.cc:454] Released schema lock
0709 13:06:18.755646 (+    42us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":1075630,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":224,"apply.run_cpu_time_us":2407,"apply.run_wall_time_us":2407,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":3039,"prepare.run_cpu_time_us":2286,"prepare.run_wall_time_us":2286,"replication_time_us":2345}]]}
I20260709 13:06:19.046157  9115 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:06:19.047775 15104 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:06:19.061283  9019 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 10 to 15, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 15 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:19.065598  9019 catalog_manager.cc:5697] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:19.072820  9189 raft_consensus.cc:1275] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:06:19.074714 15737 consensus_queue.cc:1048] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:06:19.088287  9018 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42488 (request call id 243) took 27376 ms (client timeout 15000 ms). Trace:
I20260709 13:06:19.086988  9019 catalog_manager.cc:5697] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 12 to 15, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 15 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:19.092437  9019 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 10 to 14, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 14 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
W20260709 13:06:19.088950  9018 rpcz_store.cc:269] 0709 13:05:51.711992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:05:51.712194 (+   202us) service_pool.cc:224] Handling call
0709 13:06:19.088102 (+27375908us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52195,"reported_tablets":45}
I20260709 13:06:19.095161  9019 catalog_manager.cc:5697] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 12 to 14, leader changed from 92919b70789041c8a0de0b3525154fd1 (127.7.213.68) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 14 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:19.101017  9019 catalog_manager.cc:5697] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 9 to 11, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 11 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:19.174764  9018 catalog_manager.cc:5697] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 16 to 17, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 17 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:19.258066  9019 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42488 (request call id 244) took 9067 ms. Trace:
I20260709 13:06:19.258467  9019 rpcz_store.cc:276] 0709 13:06:10.190667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.210958 (+ 20291us) service_pool.cc:224] Handling call
0709 13:06:19.257989 (+9047031us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:19.117024 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:19.117607 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:19.117642 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:06:19.117663 (+    21us) tablet.cc:662] Decoding operations
0709 13:06:19.118518 (+   855us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:19.118579 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:19.118609 (+    30us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:06:19.119174 (+   565us) tablet.cc:701] Row locks acquired
0709 13:06:19.119195 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:06:19.119337 (+   142us) write_op.cc:270] Start()
0709 13:06:19.119459 (+   122us) write_op.cc:276] Timestamp: P: 1783602379119293 usec, L: 0
0709 13:06:19.119484 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:06:19.121769 (+  2285us) log.cc:844] Serialized 2430 byte log entry
0709 13:06:19.142519 (+ 20750us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b1800102d60. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c4bfde1 00007f765c4cc5b1 00007f765a11b41d 00007f765a11d71a 00007f765a11d6a8 00007f765a11d628 00007f765a11d5e3 00007f765a11c7fc 00007f765c54f626 00007f765c549dd8 00007f76566f0310
0709 13:06:19.142571 (+    52us) op_driver.cc:464] REPLICATION: finished
0709 13:06:19.143093 (+   522us) write_op.cc:301] APPLY: starting
0709 13:06:19.143206 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:19.146713 (+  3507us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:19.146742 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:19.163197 (+ 16455us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:19.163373 (+   176us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 13:06:19.163416 (+    43us) write_op.cc:312] APPLY: finished
0709 13:06:19.163674 (+   258us) log.cc:844] Serialized 65 byte log entry
0709 13:06:19.163862 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:19.163959 (+    97us) write_op.cc:454] Released schema lock
0709 13:06:19.164002 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:19.164341 (+   339us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:19.164370 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41063,"reported_tablets":45,"spinlock_wait_cycles":25344,"child_traces":[["op",{"apply.queue_time_us":277,"apply.run_cpu_time_us":8688,"apply.run_wall_time_us":21203,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25092,"num_ops":6,"prepare.queue_time_us":110,"prepare.run_cpu_time_us":3637,"prepare.run_wall_time_us":5572,"replication_time_us":5693,"spinlock_wait_cycles":35514624}]]}
I20260709 13:06:19.267592  9012 catalog_manager.cc:5697] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 13 to 15, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 15 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:19.273082  9012 catalog_manager.cc:5697] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 14 to 15, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 15 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:19.275501  9012 catalog_manager.cc:5697] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 16 to 18, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 18 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:19.282418  9012 catalog_manager.cc:5697] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 14 to 16, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 16 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:19.287621  9012 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:19.304404  9012 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:19.389043  9192 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:19.398895  9194 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:06:19.405467 15699 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:06:19.441264  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 50801.4us
I20260709 13:06:19.497032  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:19.908164  9089 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.010s	sys 0.006s
W20260709 13:06:19.908771  9089 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.010s	sys 0.006s
W20260709 13:06:19.934767  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 529344us
I20260709 13:06:20.051532  9018 catalog_manager.cc:5697] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 11 to 12, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 12 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } }
I20260709 13:06:20.056587  9116 raft_consensus.cc:1275] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:06:20.090466 15699 consensus_queue.cc:1048] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:06:20.125269  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46903us
I20260709 13:06:20.133071 15708 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 13:06:20.203586  9859 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:06:20.371279 15708 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:06:20.401584  9260 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c199cae5ff14a478726c8a8b491f8f7 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 7 (current committed config index is 10)) from {username='slave'} at 127.0.0.1:45222
I20260709 13:06:20.408778  9259 tablet_service.cc:1558] Processing DeleteTablet for tablet 017b65b967c44e7688e66e7aab235b62 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 12)) from {username='slave'} at 127.0.0.1:45222
I20260709 13:06:20.407068  9271 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:20.441212  9012 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36634 (request call id 344) took 3588 ms. Trace:
I20260709 13:06:20.489084  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 1c199cae5ff14a478726c8a8b491f8f7 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:06:20.489956  9006 catalog_manager.cc:5028] TS 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): tablet 017b65b967c44e7688e66e7aab235b62 (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:06:20.441728  9012 rpcz_store.cc:276] 0709 13:06:16.852988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:16.855883 (+  2895us) service_pool.cc:224] Handling call
0709 13:06:20.441013 (+3585130us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:19.361906 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:19.362408 (+   502us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:19.362439 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:06:19.362452 (+    13us) tablet.cc:662] Decoding operations
0709 13:06:19.363195 (+   743us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:19.363249 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:19.363275 (+    26us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:06:19.363812 (+   537us) tablet.cc:701] Row locks acquired
0709 13:06:19.363825 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:06:19.363949 (+   124us) write_op.cc:270] Start()
0709 13:06:19.364051 (+   102us) write_op.cc:276] Timestamp: P: 1783602379363909 usec, L: 0
0709 13:06:19.364070 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:06:19.364652 (+   582us) log.cc:844] Serialized 2434 byte log entry
0709 13:06:19.366806 (+  2154us) op_driver.cc:464] REPLICATION: finished
0709 13:06:19.367272 (+   466us) write_op.cc:301] APPLY: starting
0709 13:06:19.367350 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:19.371106 (+  3756us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:19.371199 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:19.384743 (+ 13544us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:19.384956 (+   213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 13:06:19.385005 (+    49us) write_op.cc:312] APPLY: finished
0709 13:06:19.385364 (+   359us) log.cc:844] Serialized 65 byte log entry
0709 13:06:19.401177 (+ 15813us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:19.401298 (+   121us) write_op.cc:454] Released schema lock
0709 13:06:19.401348 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:19.409289 (+  7941us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:19.409328 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":87556,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":278,"apply.run_cpu_time_us":9354,"apply.run_wall_time_us":34453,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25092,"num_ops":6,"prepare.queue_time_us":120,"prepare.run_cpu_time_us":2987,"prepare.run_wall_time_us":2995,"replication_time_us":2620}]]}
I20260709 13:06:20.512511  9855 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:06:20.524367  9052 debug-util.cc:398] Leaking SignalData structure 0x7b0800359e80 after lost signal to thread 8024
W20260709 13:06:20.525841  9052 debug-util.cc:398] Leaking SignalData structure 0x7b0800232cc0 after lost signal to thread 9055
W20260709 13:06:20.527240  9052 debug-util.cc:398] Leaking SignalData structure 0x7b080008a420 after lost signal to thread 9147
W20260709 13:06:20.527518  9052 debug-util.cc:398] Leaking SignalData structure 0x7b080008a400 after lost signal to thread 9148
W20260709 13:06:20.530195  9052 debug-util.cc:398] Leaking SignalData structure 0x7b0800091bc0 after lost signal to thread 9222
W20260709 13:06:20.531401  9052 debug-util.cc:398] Leaking SignalData structure 0x7b08004162c0 after lost signal to thread 9296
W20260709 13:06:20.540759  9052 debug-util.cc:398] Leaking SignalData structure 0x7b080011b860 after lost signal to thread 9887
W20260709 13:06:20.541080  9052 debug-util.cc:398] Leaking SignalData structure 0x7b080040c2c0 after lost signal to thread 9888
I20260709 13:06:20.554761  9122 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:20.564980 15699 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:06:20.567889  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 58443.7us
I20260709 13:06:20.574216  9018 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 18 to 19, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 19 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:20.575012 15289 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:06:20.665848  9115 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:20.662568 15767 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:20.702234  9019 catalog_manager.cc:5697] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:20.725828  9857 raft_consensus.cc:2955] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:20.758689  9006 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:06:20.810986  9013 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36610 (request call id 329) took 10073 ms. Trace:
I20260709 13:06:20.811647  9013 rpcz_store.cc:276] 0709 13:06:10.737110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:10.737373 (+   263us) service_pool.cc:224] Handling call
0709 13:06:11.007172 (+269799us) spinlock_profiling.cc:243] Waited 6.54 ms on lock 0x7b540001a328. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765c8d6331 00007f765c8d13fc 00007f765c4a31b4 00007f765c5a137e 00007f765aa622a4 00007f765aa62232 00007f765aa62161 00007f765aa620dc 00007f765aa61392 00007f7654a1f48c 00007f7654a1e8c6
0709 13:06:20.808192 (+9801020us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71962,"reported_tablets":45,"spinlock_wait_cycles":15044352}
I20260709 13:06:20.851594  9019 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42488 (request call id 245) took 1494 ms. Trace:
I20260709 13:06:20.852093  9019 rpcz_store.cc:276] 0709 13:06:19.356849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:19.383661 (+ 26812us) service_pool.cc:224] Handling call
0709 13:06:20.851507 (+1467846us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:06:20.747395 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:06:20.749617 (+  2222us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:06:20.749651 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:06:20.749663 (+    12us) tablet.cc:662] Decoding operations
0709 13:06:20.750030 (+   367us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:06:20.750098 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:06:20.750132 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:06:20.755835 (+  5703us) tablet.cc:701] Row locks acquired
0709 13:06:20.755852 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:06:20.756014 (+   162us) write_op.cc:270] Start()
0709 13:06:20.756198 (+   184us) write_op.cc:276] Timestamp: P: 1783602380755968 usec, L: 0
0709 13:06:20.756219 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:06:20.756942 (+   723us) log.cc:844] Serialized 533 byte log entry
0709 13:06:20.759255 (+  2313us) op_driver.cc:464] REPLICATION: finished
0709 13:06:20.759789 (+   534us) write_op.cc:301] APPLY: starting
0709 13:06:20.759871 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:06:20.760739 (+   868us) tablet.cc:1370] finished BulkCheckPresence
0709 13:06:20.760833 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:06:20.761561 (+   728us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:06:20.761727 (+   166us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:06:20.761775 (+    48us) write_op.cc:312] APPLY: finished
0709 13:06:20.761964 (+   189us) log.cc:844] Serialized 25 byte log entry
0709 13:06:20.762114 (+   150us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:20.762200 (+    86us) write_op.cc:454] Released schema lock
0709 13:06:20.762238 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:06:20.762610 (+   372us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:06:20.762639 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":565952,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":293,"apply.run_cpu_time_us":2789,"apply.run_wall_time_us":2788,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":302,"prepare.run_cpu_time_us":2777,"prepare.run_wall_time_us":9841,"replication_time_us":2913}]]}
I20260709 13:06:20.885069  9012 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 reported cstate change: config changed from index 13 to 14, VOTER 94504d29c6744f38bb06943479df9466 (127.7.213.66) evicted. New cstate: current_term: 19 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } } }
I20260709 13:06:20.965806  9179 tablet_service.cc:1558] Processing DeleteTablet for tablet 685fa3ec557f4f3db02ec6ccce1d6f3e with delete_type TABLET_DATA_TOMBSTONED (TS 94504d29c6744f38bb06943479df9466 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:39054
I20260709 13:06:20.987851 15783 tablet_replica.cc:333] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:20.989647 15783 raft_consensus.cc:2243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:20.991037 15783 raft_consensus.cc:2272] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:21.035440 15783 ts_tablet_manager.cc:1916] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:06:21.202904 15783 ts_tablet_manager.cc:1929] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.13
I20260709 13:06:21.203455 15783 log.cc:1199] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 94504d29c6744f38bb06943479df9466: Deleting WAL directory at /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/wals/685fa3ec557f4f3db02ec6ccce1d6f3e
I20260709 13:06:21.245707  9008 catalog_manager.cc:5028] TS 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): tablet 685fa3ec557f4f3db02ec6ccce1d6f3e (table test-workload [id=d5ca54d2b4af40f2a18fb500a27c8e33]) successfully deleted
I20260709 13:06:21.404678  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:21.740005  9271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:33544
I20260709 13:06:21.740741  9271 raft_consensus.cc:606] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Received request to transfer leadership to TS 94504d29c6744f38bb06943479df9466
I20260709 13:06:21.744071  9271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:33544
I20260709 13:06:21.744652  9271 raft_consensus.cc:606] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:21.754318  9271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:33544
I20260709 13:06:21.754896  9271 raft_consensus.cc:606] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:21.782217  9267 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:33544
I20260709 13:06:21.782938  9267 raft_consensus.cc:606] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:21.788326  9267 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:33544
I20260709 13:06:21.789063  9267 raft_consensus.cc:606] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:21.793342 15760 raft_consensus.cc:993] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:21.793869 15762 raft_consensus.cc:1081] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:21.795719  9122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:06:21.796371  9122 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:21.796725  9122 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.800153  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:06:21.802008  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:21.805847  9122 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:21.817207  9122 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:21.827216  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:21.828064  9198 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.853798  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:21.854563  9267 raft_consensus.cc:3055] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:06:21.854954  9267 raft_consensus.cc:740] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:21.856074  9267 consensus_queue.cc:260] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:21.861464  9267 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:21.904374  9198 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 18.
I20260709 13:06:21.933403  9089 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:21.934652 15750 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:21.935250 15750 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:21.936269 15750 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:21.964666  9267 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 18.
I20260709 13:06:21.990636  9012 catalog_manager.cc:5697] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 17 to 18, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 18 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.072690 15762 raft_consensus.cc:993] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:22.075459 15767 raft_consensus.cc:1081] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:22.077314  9116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:06:22.077904  9116 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.078282  9116 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:22.086622  9116 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:22.094152  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:22.094707  9116 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:22.095839  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:22.096446  9862 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:22.094748  9267 raft_consensus.cc:3055] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:22.100274  9267 raft_consensus.cc:740] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:22.100991  9267 consensus_queue.cc:260] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:22.102298  9267 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:22.116003  9862 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 20.
I20260709 13:06:22.119560  9089 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:22.120795 15750 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:22.121340 15750 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 20 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:22.122352 15750 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:22.118099  9267 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 20.
I20260709 13:06:22.148392  9012 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 19 to 20, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 20 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.212821 15760 raft_consensus.cc:993] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:06:22.213376 15760 raft_consensus.cc:1081] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Signalling peer 94504d29c6744f38bb06943479df9466 to start an election
I20260709 13:06:22.215247  9198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c"
dest_uuid: "94504d29c6744f38bb06943479df9466"
 from {username='slave'} at 127.0.0.1:38858
I20260709 13:06:22.215909  9198 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.216276  9198 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:22.224555  9198 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:22.242059  9198 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:22.244029  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:22.244884  9266 raft_consensus.cc:3055] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:06:22.244964  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:22.245225  9266 raft_consensus.cc:740] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:22.245720  9862 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:22.246629  9266 consensus_queue.cc:260] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:22.247987  9266 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:06:22.253502 15762 raft_consensus.cc:993] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:22.254107 15762 raft_consensus.cc:1081] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:22.256863  9266 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 20.
I20260709 13:06:22.258487  9162 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:22.259399  9862 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 20.
I20260709 13:06:22.260411  9116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:06:22.266785  9116 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.267174  9116 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:22.273751 15760 raft_consensus.cc:993] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:22.275259 15760 raft_consensus.cc:1081] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:22.261197 15697 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:06:22.276464 15697 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 20 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:22.277426 15697 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:22.285006  9122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:06:22.285667  9122 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:22.286082  9122 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:22.307354  9116 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:22.316901  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:22.317590  9266 raft_consensus.cc:3055] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:06:22.317926  9266 raft_consensus.cc:740] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:22.318776  9266 consensus_queue.cc:260] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:22.320083  9266 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:22.320303  9116 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:22.320591  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:22.321211  9855 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:22.330662  9267 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:06:22.331840  9122 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:22.332569 15751 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:22.339378  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:22.340188  9198 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:22.344122  9266 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:06:22.335191  9855 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 12.
I20260709 13:06:22.349812  9122 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:22.360316  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:22.360967  9266 raft_consensus.cc:3055] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:06:22.361326  9266 raft_consensus.cc:740] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:22.350229  9198 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:06:22.351220  9089 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:22.365303 15751 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:22.365880 15751 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:22.371729  9266 consensus_queue.cc:260] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:22.373452  9266 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:22.366902 15751 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:22.384778  9266 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 17.
I20260709 13:06:22.399353  9012 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 19 to 20, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 20 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.418967  9087 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
W20260709 13:06:22.452747  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 62173.3us
I20260709 13:06:22.453214 15802 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:06:22.454756  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 64651.8us
W20260709 13:06:22.457746  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 66228.1us
W20260709 13:06:22.569875  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 117953us
W20260709 13:06:22.570963  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 117188us
I20260709 13:06:22.573923 15802 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:22.575124 15802 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:22.589656  9012 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 11 to 12, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 12 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.604868  9192 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:06:22.608880 15801 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:06:22.623011  9267 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:22.624544 15805 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:22.646938  9012 catalog_manager.cc:5697] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 16 to 17, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 17 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:22.667248  9855 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:06:22.671955 15802 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:22.698904  9271 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:06:22.703843 15697 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:06:22.786262  9862 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:06:22.787904 15443 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:06:22.795302  9855 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:06:22.797049 15802 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:22.803136  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:22.872370  9271 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:06:22.881870 15803 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:06:23.040773  9198 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:23.042505 15802 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:23.141149  9271 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:23.146770 15804 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:06:24.158253  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:06:24.400560  9267 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5"
dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
mode: GRACEFUL
new_leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:33544
I20260709 13:06:24.401237  9267 raft_consensus.cc:606] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 21 LEADER]: Received request to transfer leadership to TS ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:24.402740  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:06:24.404497  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:06:24.717762 15825 raft_consensus.cc:993] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: : Instructing follower ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:24.718447 15762 raft_consensus.cc:1081] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 21 LEADER]: Signalling peer ce51b61da22f40d4aa0204a6b1028f28 to start an election
I20260709 13:06:24.720386  9122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5"
dest_uuid: "ce51b61da22f40d4aa0204a6b1028f28"
 from {username='slave'} at 127.0.0.1:43780
I20260709 13:06:24.720896  9122 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:06:24.721264  9122 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:24.728169  9122 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:24.730662  9267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:24.731269  9267 raft_consensus.cc:3055] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:06:24.731532  9267 raft_consensus.cc:740] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:24.733094  9267 consensus_queue.cc:260] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:24.734295  9267 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:24.806322  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:24.809170  9862 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:24.827981  9122 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 22 election: Requested vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:24.834261  9267 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 22.
W20260709 13:06:24.837178  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 79590.2us
I20260709 13:06:24.848707  9087 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28; no voters: 
I20260709 13:06:24.850693 15801 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:24.851213 15801 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 22 LEADER]: Becoming Leader. State: Replica: ce51b61da22f40d4aa0204a6b1028f28, State: Running, Role: LEADER
I20260709 13:06:24.852169 15801 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:24.862932  9862 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce51b61da22f40d4aa0204a6b1028f28 in term 22.
I20260709 13:06:24.896798  9019 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 reported cstate change: term changed from 21 to 22, leader changed from 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67) to ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65). New cstate: current_term: 22 leader_uuid: "ce51b61da22f40d4aa0204a6b1028f28" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:25.290258  9266 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:06:25.336812 15838 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:06:25.384469  9859 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Refusing update from remote peer ce51b61da22f40d4aa0204a6b1028f28: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:06:25.386083 15801 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:06:25.405516  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:06:25.574580  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 56339.7us
I20260709 13:06:25.616969  8021 tablet_server.cc:179] TabletServer@127.7.213.65:0 shutting down...
W20260709 13:06:25.626916  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 51623.2us
W20260709 13:06:25.636801  9825 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
W20260709 13:06:25.696585  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 60597.8us
W20260709 13:06:25.721802  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63529.3us
W20260709 13:06:25.808121  9825 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:25.809578  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 80954.4us
W20260709 13:06:25.810884  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 77870.8us
W20260709 13:06:25.811688  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 116038us
W20260709 13:06:25.812773  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 165471us
W20260709 13:06:25.888849  9161 consensus_peers.cc:597] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:25.968184  9825 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:25.968940  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 135346us
W20260709 13:06:25.996876  9161 consensus_peers.cc:597] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.021874  9161 consensus_peers.cc:597] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.039918  9161 consensus_peers.cc:597] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.060037  9161 consensus_peers.cc:597] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.080788  9087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 249452us
W20260709 13:06:26.212544  9089 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 80624.2us
W20260709 13:06:26.216096  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 50718us
W20260709 13:06:26.224100  9825 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.266258  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 49156.3us
W20260709 13:06:26.267915  9825 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.317833  9235 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.324605  9235 consensus_peers.cc:597] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.325296  9235 consensus_peers.cc:597] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.319945  9161 consensus_peers.cc:597] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.341167  9161 consensus_peers.cc:597] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.348910  9161 consensus_peers.cc:597] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.353231  9161 consensus_peers.cc:597] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.353817  9235 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.356835  9161 consensus_peers.cc:597] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.361745  9235 consensus_peers.cc:597] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.363296  9235 consensus_peers.cc:597] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.362584  9161 consensus_peers.cc:597] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.371891  9235 consensus_peers.cc:597] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.394476  9235 consensus_peers.cc:597] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.399168  9235 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.403977  9235 consensus_peers.cc:597] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.405045  9235 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.405717  9235 consensus_peers.cc:597] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.402129  9825 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.413968  9825 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.416935  9825 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:26.428104  9825 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:26.555923 15801 raft_consensus.cc:493] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:26.556550 15801 raft_consensus.cc:515] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:26.558701 15801 leader_election.cc:290] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:26.560169  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:26.561386  8021 tablet_replica.cc:333] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.562441  8021 raft_consensus.cc:2243] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.563269  8021 raft_consensus.cc:2272] T 49752620cbbf430991915a2400f71286 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.566687  8021 tablet_replica.cc:333] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.567862  8021 raft_consensus.cc:2243] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.566601  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:26.568542  8021 raft_consensus.cc:2272] T aa32d6e0bb7b44ee9f664126e3ecdb72 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.570813  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cfe3483aaf54fbd932e71b72394491c" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:26.571878  8021 tablet_replica.cc:333] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.572817  8021 raft_consensus.cc:2243] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.573412  8021 raft_consensus.cc:2272] T 709565930ea044f08322d94e29d83adf P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.574756  9089 leader_election.cc:304] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:26.576598  8021 tablet_replica.cc:333] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.576535 15849 raft_consensus.cc:2749] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:26.580601  8021 raft_consensus.cc:2243] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.582484  8021 raft_consensus.cc:2272] T c0f89ac8f7b142239d6103bd6ed96037 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.585466  8021 tablet_replica.cc:333] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.586414  8021 raft_consensus.cc:2243] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.587070  8021 raft_consensus.cc:2272] T 2cfe3483aaf54fbd932e71b72394491c P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.589993  8021 tablet_replica.cc:333] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.590963  8021 raft_consensus.cc:2243] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.592000  8021 raft_consensus.cc:2272] T 9cbe8a4b1d764261adeeb8d882fa37e5 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.594748  8021 tablet_replica.cc:333] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.595629  8021 raft_consensus.cc:2243] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.596282  8021 raft_consensus.cc:2272] T 55b71b09f6834fbbbd500874ff1e14b5 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.599146  8021 tablet_replica.cc:333] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.600011  8021 raft_consensus.cc:2243] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.600816  8021 raft_consensus.cc:2272] T c07692445f814d5cab7dd13d622f9b9c P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.603510  8021 tablet_replica.cc:333] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.604305  8021 raft_consensus.cc:2243] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.605285  8021 raft_consensus.cc:2272] T 55cb8917093e4273bb3af2d4d36815cf P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.607416 15864 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:26.608032 15864 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:26.610472 15864 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:26.608022  8021 tablet_replica.cc:333] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.611024  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:26.611878  8021 raft_consensus.cc:2243] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.611792  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c199cae5ff14a478726c8a8b491f8f7" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:26.612797  8021 raft_consensus.cc:2272] T a08146adfeb9417ca7067de38da555b0 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.613437  9089 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:26.614225 15864 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:26.616922  8021 tablet_replica.cc:333] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.617782  8021 raft_consensus.cc:2243] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.618288  8021 raft_consensus.cc:2272] T 470326df6cae476c90888533e504e6ec P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.620438  8021 tablet_replica.cc:333] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.621024  8021 raft_consensus.cc:2243] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.622047  8021 raft_consensus.cc:2272] T dc3a66792b31404ba18eaacb7c54d08b P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.625655  8021 tablet_replica.cc:333] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.626925  8021 raft_consensus.cc:2243] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.627573  8021 raft_consensus.cc:2272] T 43e874ff0724409b88c5ec377b73a06a P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.631021  8021 tablet_replica.cc:333] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.631795  8021 raft_consensus.cc:2243] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.632431  8021 raft_consensus.cc:2272] T a35f9598c50046539a7f41287876135d P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.634644 15864 raft_consensus.cc:493] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:26.636435  8021 tablet_replica.cc:333] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.637385  8021 raft_consensus.cc:2243] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.637969  8021 raft_consensus.cc:2272] T 909051b50b2c464ea2b13a15a243817c P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.636003 15864 raft_consensus.cc:515] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:26.640877  8021 tablet_replica.cc:333] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.640856 15864 leader_election.cc:290] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:26.641842  8021 raft_consensus.cc:2243] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.642452  8021 raft_consensus.cc:2272] T 2b7da62e1fea4b79963197feee655d26 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.642666  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:26.643152  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a3eaf34ab444dca94a52a6fce2a556e" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:26.645236  8021 tablet_replica.cc:333] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.646394  9089 leader_election.cc:304] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
I20260709 13:06:26.647187 15864 raft_consensus.cc:2749] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:26.650424 15801 raft_consensus.cc:493] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:26.651230  8021 raft_consensus.cc:2243] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.650985 15801 raft_consensus.cc:515] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:26.652444  8021 raft_consensus.cc:2272] T db3bbfae79ea4f4cb8b7e3079bdb3908 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.655164  8021 tablet_replica.cc:333] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.655586 15801 leader_election.cc:290] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:26.655952  8021 raft_consensus.cc:2243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.656970  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:26.657739 15801 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:26.658581  8021 raft_consensus.cc:2272] T 685fa3ec557f4f3db02ec6ccce1d6f3e P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.659197 15801 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:26.661465  8021 tablet_replica.cc:333] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.662668  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:26.662350  8021 raft_consensus.cc:2243] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.663828 15801 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:26.664209  8021 raft_consensus.cc:2272] T 8a4bd074bf594b89b28b47d7482e8ac1 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.667083  8021 tablet_replica.cc:333] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.667929  8021 raft_consensus.cc:2243] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.668550  8021 raft_consensus.cc:2272] T bc89bbd2f2934bbeaf33619d3d23adfc P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.671139  8021 tablet_replica.cc:333] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.671995  8021 raft_consensus.cc:2243] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.672516  8021 raft_consensus.cc:2272] T 00ff82ce7b7342a4b052db43182f4a99 P ce51b61da22f40d4aa0204a6b1028f28 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.675629  8021 tablet_replica.cc:333] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.676436  8021 raft_consensus.cc:2243] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.677479  8021 raft_consensus.cc:2272] T a5204a9a27e54ddcb19d93142d8156f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.680112  8021 tablet_replica.cc:333] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.680917  8021 raft_consensus.cc:2243] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.681897  8021 raft_consensus.cc:2272] T 29257e9d0bc4408583c5df86d6e2fd64 P ce51b61da22f40d4aa0204a6b1028f28 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.684437  8021 tablet_replica.cc:333] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.685165  8021 raft_consensus.cc:2243] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.685681  8021 raft_consensus.cc:2272] T 9a3eaf34ab444dca94a52a6fce2a556e P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.688634  8021 tablet_replica.cc:333] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.689433  8021 raft_consensus.cc:2243] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.690496  8021 raft_consensus.cc:2272] T a6c1cc32069145ebbbacf127d2516fe8 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.692978  8021 tablet_replica.cc:333] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.693701  8021 raft_consensus.cc:2243] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.694311  8021 raft_consensus.cc:2272] T c351454182cf49938eb92870b1fb297e P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.709052  8021 tablet_replica.cc:333] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.709867  8021 raft_consensus.cc:2243] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.710525  8021 raft_consensus.cc:2272] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.721040  8021 tablet_replica.cc:333] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.721812  8021 raft_consensus.cc:2243] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.722815  8021 raft_consensus.cc:2272] T 017b65b967c44e7688e66e7aab235b62 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.725461  8021 tablet_replica.cc:333] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.725380  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c089688bfc69403095596a002ccbf890" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:26.726467  8021 raft_consensus.cc:2243] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.728080 15801 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:26.728710 15801 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:26.731290  9089 leader_election.cc:304] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:26.732251  8021 raft_consensus.cc:2272] T 776f24c8f05046a0a730da5e6e801cb7 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.732553 15801 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:26.732918  9089 raft_consensus.cc:2686] T c089688bfc69403095596a002ccbf890 P ce51b61da22f40d4aa0204a6b1028f28: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:26.731371  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58cbd03222246299b7e4363d8fcffa6" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:26.735268  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:26.736433  8021 tablet_replica.cc:333] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.737313  8021 raft_consensus.cc:2243] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.737862  8021 raft_consensus.cc:2272] T d8f97d26e68b4fbead130c830dbf4b51 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.740744  8021 tablet_replica.cc:333] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.741516  8021 raft_consensus.cc:2243] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.742481  8021 raft_consensus.cc:2272] T 991b28944986479a8ad096ca92f1a435 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.744179 15801 raft_consensus.cc:493] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 92919b70789041c8a0de0b3525154fd1)
I20260709 13:06:26.744798 15801 raft_consensus.cc:515] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:26.745420  8021 tablet_replica.cc:333] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.746243  8021 raft_consensus.cc:2243] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.747191  8021 raft_consensus.cc:2272] T 9daa912f9e08438caf063c0c83e09fe5 P ce51b61da22f40d4aa0204a6b1028f28 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.747716 15801 leader_election.cc:290] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:26.748175  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d9ec1fba08a49aeb3e1a8a52e798291" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:26.750239  8021 tablet_replica.cc:333] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.750928  9089 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:26.751097  8021 raft_consensus.cc:2243] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.751073  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:26.752660  8021 raft_consensus.cc:2272] T 1c199cae5ff14a478726c8a8b491f8f7 P ce51b61da22f40d4aa0204a6b1028f28 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.752978 15801 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:26.756042  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff8d71c72ec64addaf0593eca7fd672d" candidate_uuid: "ce51b61da22f40d4aa0204a6b1028f28" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:26.758917  9089 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1
I20260709 13:06:26.759622 15801 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:26.761838  8021 tablet_replica.cc:333] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.763587  8021 raft_consensus.cc:2243] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.765436  8021 raft_consensus.cc:2272] T 8d9ec1fba08a49aeb3e1a8a52e798291 P ce51b61da22f40d4aa0204a6b1028f28 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.768155  8021 tablet_replica.cc:333] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.768977  8021 raft_consensus.cc:2243] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.769565  8021 raft_consensus.cc:2272] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.771047  9089 leader_election.cc:304] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce51b61da22f40d4aa0204a6b1028f28; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:06:26.772907  9089 raft_consensus.cc:2686] T ff8d71c72ec64addaf0593eca7fd672d P ce51b61da22f40d4aa0204a6b1028f28: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:06:26.777377  8021 tablet_replica.cc:333] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.778110  8021 raft_consensus.cc:2243] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.778973  8021 raft_consensus.cc:2272] T 7c3c420051574414bbec6a77f8036b61 P ce51b61da22f40d4aa0204a6b1028f28 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.781936  8021 tablet_replica.cc:333] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.782707  8021 raft_consensus.cc:2243] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.783986  8021 raft_consensus.cc:2272] T 7597d602dc674623ae1d5de5fdb5d2ee P ce51b61da22f40d4aa0204a6b1028f28 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.786849  8021 tablet_replica.cc:333] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.787562  8021 raft_consensus.cc:2243] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.788264  8021 raft_consensus.cc:2272] T 23636a3a86df4bdf91dc6a7ffe723527 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.790865  8021 tablet_replica.cc:333] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.796844  8021 raft_consensus.cc:2243] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.797837  8021 raft_consensus.cc:2272] T bd9140502bd3469eb208753dea0f3159 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.804663  8021 tablet_replica.cc:333] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.805405  8021 raft_consensus.cc:2243] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.806025  8021 raft_consensus.cc:2272] T 90c9bcfd54594ec58c5e8e1be6c556d8 P ce51b61da22f40d4aa0204a6b1028f28 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.808934  8021 tablet_replica.cc:333] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.809628  8021 raft_consensus.cc:2243] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.810696  8021 raft_consensus.cc:2272] T b0917335c1234d5abec1e40c742de6be P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.813651  8021 tablet_replica.cc:333] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.814389  8021 raft_consensus.cc:2243] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.815493  8021 raft_consensus.cc:2272] T dfa5110f517a4ffb866953379f5d0c4d P ce51b61da22f40d4aa0204a6b1028f28 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.819339  8021 tablet_replica.cc:333] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.821328  8021 raft_consensus.cc:2243] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.822132  8021 raft_consensus.cc:2272] T e58cbd03222246299b7e4363d8fcffa6 P ce51b61da22f40d4aa0204a6b1028f28 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.828459  8021 tablet_replica.cc:333] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.829149  8021 raft_consensus.cc:2243] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:26.835628  8021 raft_consensus.cc:2272] T a51a0795b52e4843959cd3be3d077713 P ce51b61da22f40d4aa0204a6b1028f28 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:26.850175  8021 tablet_replica.cc:333] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28: stopping tablet replica
I20260709 13:06:26.850973  8021 raft_consensus.cc:2243] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:26.852145  8021 raft_consensus.cc:2272] T ca0b93b18a7f4e6398018531838527d9 P ce51b61da22f40d4aa0204a6b1028f28 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:27.742494 15869 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.743105 15869 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:27.745353 15869 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:27.748303  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
W20260709 13:06:27.752912  9825 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:27.753472  9825 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:27.754752 15869 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:27.817742 15877 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.818467 15877 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:27.823650 15885 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.824375 15885 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:27.828225 15880 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.828869 15880 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:27.829375 15862 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.829986 15862 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:27.831377 15880 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:27.832168 15862 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:27.833971 15846 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.850131 15846 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:27.852537 15846 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:27.852842 15862 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.853502 15862 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:27.833966  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:27.856089  9264 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 17.
I20260709 13:06:27.837589  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:27.857285  9197 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
I20260709 13:06:27.838245  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:27.858877  9192 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:27.861672 15885 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:27.844437 15877 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:27.845835  9161 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:27.880724 15869 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.881506 15869 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:27.888085  9161 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:27.889233  9825 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:27.890511 15869 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:27.896184 15846 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.897017 15846 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:27.899636 15846 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:27.900743 15880 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.901880 15880 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:27.904639 15880 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:27.905835  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:27.909499 15862 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:27.913684  9235 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:27.916263 15890 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.918380  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:27.931025 15890 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:27.931528  9859 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 20.
I20260709 13:06:27.935009 15869 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.935603 15869 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:27.917764 15877 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.919953  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:27.921598  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:27.940899 15877 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:27.945107 15890 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:27.945713 15877 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:27.946753  9825 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:27.948809 15869 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:27.950053  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:27.956265  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:27.957018  9264 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 13.
I20260709 13:06:27.960693  9162 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:27.964401 15880 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:27.921845  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:27.964886 15880 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:06:27.966506  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 44759.6us
I20260709 13:06:27.967376  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:27.966580 15880 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:27.969388 15888 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:27.971397 15888 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:27.971923 15888 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:27.970911 15890 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:27.975028 15890 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:06:27.976897  9235 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:27.975517 15890 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:27.985643  9825 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:27.986981  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:27.989471  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:27.990162  9271 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 20.
I20260709 13:06:27.990276  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:27.990968  9263 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 12.
I20260709 13:06:27.995843  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:27.996476  9857 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
W20260709 13:06:27.998435  9235 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.001631 15887 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:28.002174 15887 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.002573 15887 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:28.008354  9161 leader_election.cc:336] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.008960  9235 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.009692  9235 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.013047 15891 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.013674 15891 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.022562  9238 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.024659 15869 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.025327 15869 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.029500  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:28.008919  9161 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.030877 15869 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:28.031924 15886 raft_consensus.cc:2749] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:06:28.043869  9825 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.044905 15890 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.045472 15886 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.046135 15886 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.047214 15890 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:28.048473  9161 leader_election.cc:336] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.049042  9161 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.051007 15892 raft_consensus.cc:2749] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:28.053184 15886 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:28.054428  9235 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.055972 15862 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:28.056424 15862 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.056828 15862 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:28.061105 15877 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.061870 15877 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.063496  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:28.066103 15846 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.066751 15846 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.069375 15846 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:28.069938 15859 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.064649 15877 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.070556 15859 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.072110  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:28.073501 15859 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.044456  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.075202 15893 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.078408 15869 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:28.078859  8021 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 9144)
W20260709 13:06:28.067368  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 44915.6us
I20260709 13:06:28.079409 15893 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.083736  9238 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.084843 15846 raft_consensus.cc:2749] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:28.085187  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:28.085875  9192 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 20.
I20260709 13:06:28.086084  9238 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.086156 15893 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:28.087056 15846 raft_consensus.cc:2749] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:28.082937 15891 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:28.087437  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:28.089141 15869 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:28.089609 15869 raft_consensus.cc:493] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.090026 15869 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:28.090888 15893 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.091473 15893 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.092788  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.078019 15888 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.098613 15888 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:28.030900 15880 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.101652 15880 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.102737 15862 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.103395  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:28.104367  9271 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 18: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:28.111919 15893 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:28.114635 15887 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.117512 15846 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:28.118033 15846 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.118499 15846 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:28.124106  9161 leader_election.cc:336] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.124748  9161 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.125737 15893 raft_consensus.cc:2749] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:28.124004 15869 raft_consensus.cc:515] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:28.126786  9161 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.127265  9161 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.128073 15893 raft_consensus.cc:2749] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:28.130790 15888 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.131443 15888 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.132131 15889 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.132754 15889 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.133152  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29257e9d0bc4408583c5df86d6e2fd64" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:28.133626 15888 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:28.133796  9192 raft_consensus.cc:3060] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:06:28.134860  9161 consensus_peers.cc:597] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:28.135504  9161 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.135535 15889 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:28.136451  9161 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.137035  9161 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.137656  9161 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.138455 15893 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:06:28.145148  9192 raft_consensus.cc:2468] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 21.
I20260709 13:06:28.146087 15869 leader_election.cc:290] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:28.148411  9828 leader_election.cc:304] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:28.149734 15862 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:28.150607 15859 raft_consensus.cc:2804] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:28.151242 15859 raft_consensus.cc:697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 21 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:28.152281 15859 consensus_queue.cc:237] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.154371 15869 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.155061 15869 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.155689 15893 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.156307 15893 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.158820  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:28.159483  9857 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 18.
I20260709 13:06:28.160085  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:28.160218 15846 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.162334 15893 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.164300  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:28.165175  9271 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 14: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:28.167449 15869 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.170768  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:28.192416  9862 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 20.
W20260709 13:06:28.178661  9161 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.193938  9161 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.183934 15846 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:28.146674  9235 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.188962  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:28.195225  9825 leader_election.cc:336] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.189780  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:28.180073 15887 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:28.197897  9235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 51264.7us
I20260709 13:06:28.198073  9264 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:28.196575 15895 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
W20260709 13:06:28.197466  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 46131us
W20260709 13:06:28.198509  9825 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.198742  9266 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 14.
W20260709 13:06:28.195819  9161 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.200111  9235 leader_election.cc:336] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.206418  9825 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.197443 15893 raft_consensus.cc:2749] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:28.208199  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:28.215494  9235 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.217267  9161 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.218252 15877 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:06:28.221174  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:28.221907  9197 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 14: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:28.224613  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:28.225577  9197 raft_consensus.cc:2393] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 18: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:28.230724 15877 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:28.231171 15877 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.231537 15877 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:28.232573  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.234972 15880 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:28.235378 15880 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.235767 15880 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:28.240314 15877 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.244598 15877 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:28.246920  9235 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.247617  9235 leader_election.cc:336] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.205646 15895 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.252822  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:28.253652  9197 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 18.
I20260709 13:06:28.253990 15895 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:28.254841  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 85708.4us
W20260709 13:06:28.256335  9235 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.256409  9826 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:28.257074  9235 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.257820 15859 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:06:28.258100  9235 consensus_peers.cc:597] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.258714 15859 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.259584 15859 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:28.260418  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:28.261288  9197 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 16.
I20260709 13:06:28.261340  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.263177 15898 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:28.263643 15898 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.264056 15898 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:28.268378  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.270220 15897 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:28.270642 15897 raft_consensus.cc:493] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.271030 15897 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:28.272807  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:06:28.272905  9161 consensus_peers.cc:597] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.274477 15869 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:28.274889 15869 raft_consensus.cc:493] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.275300 15869 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:28.275005 15859 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.277483 15859 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.276954 15880 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.284054 15869 raft_consensus.cc:515] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.285616 15895 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.286199 15895 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.288306 15895 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:28.289330 15869 leader_election.cc:290] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.317348  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:28.317664  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:28.318253  9857 raft_consensus.cc:2393] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 15: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:28.318413  9859 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 21: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
I20260709 13:06:28.319566  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:28.324137  9019 catalog_manager.cc:5697] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 20 to 21, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 21 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:28.333262  9825 leader_election.cc:336] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.336187  9161 consensus_peers.cc:597] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:28.336676  9161 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.341521  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:28.342175  9192 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 17.
I20260709 13:06:28.344411 15885 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.344970 15885 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.347182 15885 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:28.347398  9161 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.348932 15880 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:28.351655 15897 raft_consensus.cc:515] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.353284  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c1cc32069145ebbbacf127d2516fe8" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:06:28.353838  9235 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.354660 15898 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.353940  9857 raft_consensus.cc:3060] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:28.355969  9859 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 13: Already voted for candidate 92919b70789041c8a0de0b3525154fd1 in this term.
W20260709 13:06:28.360492  9161 consensus_peers.cc:597] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:28.355137  9825 leader_election.cc:336] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.364418  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:28.365964 15899 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:28.366463 15899 raft_consensus.cc:493] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.366887 15899 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:28.373901  9857 raft_consensus.cc:2468] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 19.
I20260709 13:06:28.378592 15899 raft_consensus.cc:515] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.347095  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:28.383145  9825 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.383318  9164 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.383939  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0917335c1234d5abec1e40c742de6be" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:28.384831  9189 raft_consensus.cc:3060] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:28.385329  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.384833 15898 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.354020 15897 leader_election.cc:290] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.386065 15880 raft_consensus.cc:2804] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:28.390498 15880 raft_consensus.cc:697] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 19 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:06:28.390669  9825 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.382138 15899 leader_election.cc:290] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:28.384583 15862 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:06:28.391577  9825 leader_election.cc:336] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.394753  9825 leader_election.cc:336] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.396634 15862 raft_consensus.cc:2749] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:06:28.399569  9235 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.400439  9857 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db3bbfae79ea4f4cb8b7e3079bdb3908" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:28.401085  9857 raft_consensus.cc:3060] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:28.391647 15880 consensus_queue.cc:237] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:28.409057  9161 consensus_peers.cc:597] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:28.411645  9825 leader_election.cc:336] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.412434  9825 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.412916  9825 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.414892  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.416260  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.417361  9857 raft_consensus.cc:2468] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 21.
I20260709 13:06:28.419062  9189 raft_consensus.cc:2468] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:06:28.419153  9164 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.420009 15877 raft_consensus.cc:2804] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:28.420717  9828 leader_election.cc:304] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:28.420965 15885 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:06:28.421357 15885 raft_consensus.cc:493] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.421628 15862 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:28.421732 15885 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:28.422077 15862 raft_consensus.cc:493] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.422482 15862 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:28.425072  9161 consensus_peers.cc:597] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.431249 15897 raft_consensus.cc:2804] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:06:28.433772  9825 leader_election.cc:336] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.439009  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:28.439918  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55cb8917093e4273bb3af2d4d36815cf" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:28.431857 15897 raft_consensus.cc:697] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:28.442971 15897 consensus_queue.cc:237] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.440577  9264 raft_consensus.cc:3060] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:28.445261  9270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:28.446316  9270 raft_consensus.cc:2393] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 13: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:28.439006  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:28.446409  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b93b18a7f4e6398018531838527d9" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:28.448232  9271 raft_consensus.cc:2393] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 92919b70789041c8a0de0b3525154fd1 in current term 21: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
I20260709 13:06:28.448314  9266 raft_consensus.cc:3060] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:28.439937  9263 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 22.
I20260709 13:06:28.460165  9266 raft_consensus.cc:2468] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
I20260709 13:06:28.464480  9264 raft_consensus.cc:2468] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 15.
W20260709 13:06:28.469590  9235 consensus_peers.cc:597] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:28.473493  9161 consensus_peers.cc:597] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.420727 15877 raft_consensus.cc:697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 21 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
W20260709 13:06:28.484635  9825 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.487879 15877 consensus_queue.cc:237] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.495555  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:28.496256  9189 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:28.507309  9238 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.512791  9826 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.513841  9238 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.492620 15885 raft_consensus.cc:515] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.517689  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.517346 15891 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:28.526283 15894 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:28.516436 15889 raft_consensus.cc:2749] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:28.441352 15862 raft_consensus.cc:515] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.531867 15885 leader_election.cc:290] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:28.533502 15862 leader_election.cc:290] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:28.563925  9235 leader_election.cc:336] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.564484  9235 leader_election.cc:336] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.546173 15862 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:06:28.567116  9192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dfa5110f517a4ffb866953379f5d0c4d" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:28.567785  9192 raft_consensus.cc:3060] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:06:28.572103  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "776f24c8f05046a0a730da5e6e801cb7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:28.572752  9189 raft_consensus.cc:3060] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:28.574021  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.582762  9192 raft_consensus.cc:2468] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
I20260709 13:06:28.584940  9189 raft_consensus.cc:2468] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 19.
I20260709 13:06:28.595948  9238 leader_election.cc:304] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.603251  9238 leader_election.cc:304] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.604897 15862 raft_consensus.cc:2804] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:06:28.605449 15862 raft_consensus.cc:697] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:28.606343 15862 consensus_queue.cc:237] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.606053  9019 catalog_manager.cc:5697] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 18 to 19, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 19 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:28.608107 15885 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:28.608564 15885 raft_consensus.cc:493] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.609176 15885 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:28.609762  9189 raft_consensus.cc:1275] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:06:28.608256 15894 raft_consensus.cc:2804] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:28.616650 15894 raft_consensus.cc:697] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:28.617554 15894 consensus_queue.cc:237] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:28.622632  9013 catalog_manager.cc:5697] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 17 to 18, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 18 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:28.626078  9235 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.630259 15885 raft_consensus.cc:515] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:28.635697  9235 consensus_peers.cc:597] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.635864 15885 leader_election.cc:290] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:28.636320  9235 consensus_peers.cc:597] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:28.637703  9161 consensus_peers.cc:597] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.638912  9189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c3c420051574414bbec6a77f8036b61" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:28.639487  9189 raft_consensus.cc:3060] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:06:28.640731  9235 leader_election.cc:336] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:28.643533  9826 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 125558us
I20260709 13:06:28.644233  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.645880  9826 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.646759 15907 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:06:28.647245 15907 raft_consensus.cc:493] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader ce51b61da22f40d4aa0204a6b1028f28)
I20260709 13:06:28.647616 15907 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:28.615374 15897 consensus_queue.cc:1048] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
W20260709 13:06:28.652901  9825 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:28.653402  9825 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.658756  9826 leader_election.cc:304] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.660760  9826 leader_election.cc:304] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.661849  9189 raft_consensus.cc:2468] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:28.663947  9238 leader_election.cc:304] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:28.660257 15895 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:06:28.664875 15897 raft_consensus.cc:2804] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:28.669384 15889 raft_consensus.cc:2804] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:28.670007 15889 raft_consensus.cc:697] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:28.670351 15907 raft_consensus.cc:515] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.670967 15889 consensus_queue.cc:237] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:28.683501  9161 consensus_peers.cc:597] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:28.675117  9235 consensus_peers.cc:597] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.676106 15898 raft_consensus.cc:2804] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:06:28.684899 15898 raft_consensus.cc:697] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:28.685830 15898 consensus_queue.cc:237] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:28.680215  9825 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:28.714241  9825 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.675154 15897 raft_consensus.cc:697] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:28.715328 15897 consensus_queue.cc:237] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.738852  9019 catalog_manager.cc:5697] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 20 to 21, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 21 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:28.736625  9013 catalog_manager.cc:5697] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 16 to 17, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 17 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:28.750257  9235 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.757513  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9daa912f9e08438caf063c0c83e09fe5" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:28.758172  9264 raft_consensus.cc:3060] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:06:28.758549 15907 leader_election.cc:290] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:28.776024  9264 raft_consensus.cc:2468] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 23.
I20260709 13:06:28.777127  9826 leader_election.cc:304] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:28.781229 15907 raft_consensus.cc:2804] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:06:28.783895 15907 raft_consensus.cc:697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 23 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
W20260709 13:06:28.803422  9825 leader_election.cc:336] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:28.808403 15907 consensus_queue.cc:237] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:28.805961  9012 catalog_manager.cc:5697] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 17 to 18, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 18 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:28.808925  9161 consensus_peers.cc:597] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.811257  9012 catalog_manager.cc:5697] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 14 to 15, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 15 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.817616  9013 catalog_manager.cc:5697] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 18 to 19, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 19 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:28.819588  9013 catalog_manager.cc:5697] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 13 to 14, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 14 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:28.821138  9235 consensus_peers.cc:597] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:28.841547  9825 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:28.909406  9857 raft_consensus.cc:1275] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:06:28.920540 15918 consensus_queue.cc:1048] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:06:28.934732  9161 consensus_peers.cc:597] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:28.935452  9235 consensus_peers.cc:597] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:28.931761  9013 catalog_manager.cc:5697] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 22 to 23, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 23 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:06:28.947223  9161 consensus_peers.cc:597] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:29.010669  9198 raft_consensus.cc:1275] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:06:29.012408 15862 consensus_queue.cc:1048] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:06:29.075941  9235 consensus_peers.cc:597] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:29.080979  8021 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 9144)
I20260709 13:06:29.099481  9189 raft_consensus.cc:1275] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:29.101771  9857 raft_consensus.cc:1275] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:06:29.105463  9189 raft_consensus.cc:1275] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:06:29.107164 15862 consensus_queue.cc:1048] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:06:29.119841 15917 consensus_queue.cc:1048] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:06:29.121626 15898 consensus_queue.cc:1048] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:06:29.145922  9235 consensus_peers.cc:597] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:29.146502  9235 consensus_peers.cc:597] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:29.160228  9264 raft_consensus.cc:1275] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:06:29.163177 15898 consensus_queue.cc:1048] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:06:29.205461  9189 raft_consensus.cc:1275] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:06:29.208860 15862 consensus_queue.cc:1048] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:06:29.211848  9235 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:29.307976  9264 raft_consensus.cc:1275] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:29.309600 15898 consensus_queue.cc:1048] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:06:29.387691  9235 consensus_peers.cc:597] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:29.413694  9825 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:29.414263  9825 consensus_peers.cc:597] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:29.430084  9161 consensus_peers.cc:597] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:29.518623  9264 raft_consensus.cc:1275] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 23 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:06:29.521492 15895 consensus_queue.cc:1048] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:06:29.538990  9825 consensus_peers.cc:597] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:29.571333  9825 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:29.816428  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 63140.3us
W20260709 13:06:29.860121  9164 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 61457.7us
I20260709 13:06:29.908368 15933 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.909505 15933 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:29.928856  9235 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:29.930554 15933 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:29.944290 15918 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:29.945089 15918 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:29.949616  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:29.958586  9197 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:29.966075 15918 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:29.967857  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:29.968551  9266 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 14.
I20260709 13:06:29.969926  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:29.970901 15934 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:29.971414 15934 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.971875 15934 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:06:29.976466  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:29.978428 15862 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:06:29.978819 15862 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:29.979118 15862 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:06:29.986874  9161 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:29.994451 15862 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:30.001883 15934 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:30.013572 15934 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:30.013921  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:30.014766  9266 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 94504d29c6744f38bb06943479df9466 in current term 15: Already voted for candidate 1efabcb1befc400aadf101e8276fc1aa in this term.
W20260709 13:06:30.003307  9235 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:30.025077  9161 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:30.025651  9161 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94504d29c6744f38bb06943479df9466; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:30.027698 15934 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:30.040038 15888 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.040856 15888 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:30.043455 15888 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:30.045210  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:30.045958  9859 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 21.
I20260709 13:06:30.047401  9238 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:30.049566 15888 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:06:30.050030 15888 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.050496 15888 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:30.050107 15862 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:30.056375  9235 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:30.058252  9197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:30.059405  9197 raft_consensus.cc:2393] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1efabcb1befc400aadf101e8276fc1aa in current term 15: Already voted for candidate 94504d29c6744f38bb06943479df9466 in this term.
I20260709 13:06:30.061486  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:30.062413 15862 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:06:30.066380 15888 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:30.081617  8021 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 9144)
W20260709 13:06:30.094187  9235 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:30.095227 15888 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 22 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:30.096287  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "685fa3ec557f4f3db02ec6ccce1d6f3e" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:30.096966  9859 raft_consensus.cc:3060] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:06:30.105249  9859 raft_consensus.cc:2468] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 22.
I20260709 13:06:30.111829  9238 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:30.112692 15888 raft_consensus.cc:2804] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:06:30.113227 15888 raft_consensus.cc:697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 22 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:30.114467 15888 consensus_queue.cc:237] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:30.154309  9019 catalog_manager.cc:5697] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 20 to 22, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 22 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:30.209147 15862 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.209820 15862 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:30.222160 15862 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:30.224337  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:30.225116  9859 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:30.226342  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
W20260709 13:06:30.227059  9235 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:30.227268 15862 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:06:30.227689 15862 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.228057 15862 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:30.250691 15862 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:30.255882 15862 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:30.256846  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbe8a4b1d764261adeeb8d882fa37e5" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
W20260709 13:06:30.261950  9235 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:30.262449  9859 raft_consensus.cc:3060] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:06:30.271479  9859 raft_consensus.cc:2468] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 14.
I20260709 13:06:30.278534  9238 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:30.280059 15862 raft_consensus.cc:2804] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:06:30.280679 15862 raft_consensus.cc:697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:30.281822 15862 consensus_queue.cc:237] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:30.334798 15888 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:30.335466 15888 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:30.341899 15888 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:30.342881  9235 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:30.372927  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466" is_pre_election: true
I20260709 13:06:30.373607  9198 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 18.
I20260709 13:06:30.375172  9019 catalog_manager.cc:5697] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 12 to 14, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 14 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:30.386812  9825 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:30.402173  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:30.408677 15933 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:06:30.409214 15933 raft_consensus.cc:493] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:30.409639 15933 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:06:30.421691  9161 consensus_peers.cc:597] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:30.431375  9825 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:30.462623 15933 raft_consensus.cc:515] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:30.471423  9235 leader_election.cc:336] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:30.477221  9198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc3a66792b31404ba18eaacb7c54d08b" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "94504d29c6744f38bb06943479df9466"
I20260709 13:06:30.477877  9198 raft_consensus.cc:3060] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:06:30.482626 15933 leader_election.cc:290] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:30.488400  9198 raft_consensus.cc:2468] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 19.
I20260709 13:06:30.491218  9238 leader_election.cc:304] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:30.497478 15888 raft_consensus.cc:2804] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:06:30.498025 15888 raft_consensus.cc:697] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:30.499058 15888 consensus_queue.cc:237] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:30.569159  9019 catalog_manager.cc:5697] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 17 to 19, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 19 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:30.617525  9235 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:30.661511  9859 raft_consensus.cc:1275] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:06:30.663646 15943 consensus_queue.cc:1048] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:06:30.671250  9161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111) [suppressed 390 similar messages]
W20260709 13:06:30.712460  9161 consensus_peers.cc:597] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:30.727171  9161 consensus_peers.cc:597] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:30.763640  9855 raft_consensus.cc:1275] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:06:30.766424 15933 consensus_queue.cc:1048] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:06:30.770928  9161 consensus_peers.cc:597] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:30.778297  9235 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:30.808708  9235 consensus_peers.cc:597] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:30.849303  9235 consensus_peers.cc:597] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:30.853943  9161 consensus_peers.cc:597] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:30.877048  9161 consensus_peers.cc:597] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:30.895045  9825 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:30.952168  9198 raft_consensus.cc:1275] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:06:30.959571 15933 consensus_queue.cc:1048] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:06:30.966502  9161 consensus_peers.cc:597] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:30.969492  9235 consensus_peers.cc:597] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.001471  9161 consensus_peers.cc:597] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.003371  9235 consensus_peers.cc:597] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:31.057484  9825 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.088634  8021 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 9144)
W20260709 13:06:31.097664  9161 consensus_peers.cc:597] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.127274  9825 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:31.179240  9825 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.193504  9235 consensus_peers.cc:597] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.212872  9161 consensus_peers.cc:597] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.216063  9235 consensus_peers.cc:597] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.242139  9235 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.242877  9825 consensus_peers.cc:597] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:31.247309  9235 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.315526  9161 consensus_peers.cc:597] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:31.387920  9825 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.411803  9825 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.458267  9235 consensus_peers.cc:597] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.534988  9235 consensus_peers.cc:597] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:31.600749  9162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:38807, user_credentials={real_user=slave}} blocked reactor thread for 141071us
W20260709 13:06:31.602516  9825 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:31.603089  9235 consensus_peers.cc:597] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:31.610179  9235 consensus_peers.cc:597] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.623529  9161 consensus_peers.cc:597] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:31.625036  9235 consensus_peers.cc:597] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.639791  9235 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.652019  9235 consensus_peers.cc:597] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.697086  9825 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:31.707661  9235 consensus_peers.cc:597] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:31.720649  9825 consensus_peers.cc:597] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:31.924211  9161 consensus_peers.cc:597] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:32.059943 15954 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:32.060506 15954 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:32.063083 15954 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:32.068715  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:32.069471  9271 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 15.
W20260709 13:06:32.075254  9825 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:32.081104  9161 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:32.083338  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:32.084919 15956 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:06:32.085383 15956 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:32.088374 15956 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:06:32.089130  8021 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 9144)
I20260709 13:06:32.110790 15956 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:32.122787 15956 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Requested vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:32.123934  9271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5204a9a27e54ddcb19d93142d8156f7" candidate_uuid: "94504d29c6744f38bb06943479df9466" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:32.124647  9271 raft_consensus.cc:3060] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:06:32.132754  9271 raft_consensus.cc:2468] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94504d29c6744f38bb06943479df9466 in term 16.
I20260709 13:06:32.140888  9162 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466; no voters: 
I20260709 13:06:32.144212 15934 raft_consensus.cc:2804] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:06:32.145056 15934 raft_consensus.cc:697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Becoming Leader. State: Replica: 94504d29c6744f38bb06943479df9466, State: Running, Role: LEADER
I20260709 13:06:32.146148 15934 consensus_queue.cc:237] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:32.188360  9161 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:32.251246  9019 catalog_manager.cc:5697] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 reported cstate change: term changed from 13 to 16, leader changed from ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65) to 94504d29c6744f38bb06943479df9466 (127.7.213.66). New cstate: current_term: 16 leader_uuid: "94504d29c6744f38bb06943479df9466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:32.552157  9161 consensus_peers.cc:597] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:32.714861  9271 raft_consensus.cc:1275] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Refusing update from remote peer 94504d29c6744f38bb06943479df9466: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:06:32.717474 15956 consensus_queue.cc:1048] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:06:32.914987  9161 consensus_peers.cc:597] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:32.952121  9825 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.005681  9825 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.079783  9161 consensus_peers.cc:597] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.089766  8021 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 9144)
W20260709 13:06:33.103217  9235 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.111215  9825 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.135413  9235 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:33.172259  9235 consensus_peers.cc:597] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:33.237309  9161 consensus_peers.cc:597] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.246917  9235 consensus_peers.cc:597] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.312968  9825 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.349982  9235 consensus_peers.cc:597] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:33.382304  9161 consensus_peers.cc:597] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.435635  9161 consensus_peers.cc:597] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.467761  9161 consensus_peers.cc:597] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.468264  9161 consensus_peers.cc:597] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.553774  9235 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.611158  9235 consensus_peers.cc:597] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.616921  9825 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:33.619186  9161 consensus_peers.cc:597] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:33.636688  9235 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.639345  9161 consensus_peers.cc:597] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.674520  9825 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.694608  9161 consensus_peers.cc:597] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.695950  9825 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.720875  9161 consensus_peers.cc:597] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.871068  9235 consensus_peers.cc:597] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.876402  9235 consensus_peers.cc:597] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:33.885811  9235 consensus_peers.cc:597] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:33.926973  9825 consensus_peers.cc:597] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:33.934223  9235 consensus_peers.cc:597] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:34.087195  9825 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:34.090286  8021 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 9144)
W20260709 13:06:34.118633  9235 consensus_peers.cc:597] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:34.120420  9825 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:34.145354  9235 consensus_peers.cc:597] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:34.152983  9235 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:34.163000  9235 consensus_peers.cc:597] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:34.184060  9235 consensus_peers.cc:597] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:34.187126  9161 consensus_peers.cc:597] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:34.203881  9825 consensus_peers.cc:597] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:34.218073  9825 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:34.381989  9825 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:34.411805  9235 consensus_peers.cc:597] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:34.578756  9161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 44242.5us
W20260709 13:06:34.931146  9161 consensus_peers.cc:597] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:35.091626  8021 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 9144)
W20260709 13:06:35.101505  9161 consensus_peers.cc:597] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:35.465867  9825 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:35.651533  9235 consensus_peers.cc:597] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:35.669039  9161 consensus_peers.cc:597] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:35.690698  9825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111) [suppressed 442 similar messages]
W20260709 13:06:35.694567  9825 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:35.714089  9161 consensus_peers.cc:597] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:35.754169  9235 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:35.775897  9825 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:35.796584  9235 consensus_peers.cc:597] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:35.868376  9235 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:35.915419  9825 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:35.944180  9235 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:35.975391  9161 consensus_peers.cc:597] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.028963  9235 consensus_peers.cc:597] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:36.058066  9161 consensus_peers.cc:597] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.088213  9161 consensus_peers.cc:597] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.093868  8021 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 9144)
W20260709 13:06:36.102363  9161 consensus_peers.cc:597] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.109942  9235 consensus_peers.cc:597] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.114521  9825 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:36.116717  9825 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.193784  9235 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.204524  9161 consensus_peers.cc:597] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.238703  9161 consensus_peers.cc:597] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.284298  9825 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.342309  9235 consensus_peers.cc:597] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.363546  9161 consensus_peers.cc:597] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:36.365238  9235 consensus_peers.cc:597] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:36.424016  9235 consensus_peers.cc:597] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.441932  9825 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:36.477952  9161 consensus_peers.cc:597] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.491928  9161 consensus_peers.cc:597] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:36.528716  9161 consensus_peers.cc:597] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.558003  9235 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.582525  9235 consensus_peers.cc:597] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.630822  9825 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:36.640465  9235 consensus_peers.cc:597] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.713294  9235 consensus_peers.cc:597] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.722982  9825 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.758924  9825 consensus_peers.cc:597] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:36.802248  9235 consensus_peers.cc:597] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:36.804994  9235 consensus_peers.cc:597] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:36.951227  9235 consensus_peers.cc:597] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:36.979388  9825 consensus_peers.cc:597] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:37.030510  9825 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:37.154865  8021 tablet_server.cc:196] TabletServer@127.7.213.65:0 shutdown complete.
W20260709 13:06:37.369179  9161 consensus_peers.cc:597] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:37.385506  9161 consensus_peers.cc:597] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:37.542925  8021 tablet_server.cc:179] TabletServer@127.7.213.66:0 shutting down...
W20260709 13:06:37.550812  9238 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.553035  9828 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.565191  9828 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.600757  9238 consensus_peers.cc:597] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.636288  9238 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.672961  9828 consensus_peers.cc:597] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.696929  9828 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.697705  9828 consensus_peers.cc:597] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.700196  9238 consensus_peers.cc:597] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.711494  9238 consensus_peers.cc:597] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.720477  9828 consensus_peers.cc:597] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:37.730319  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:37.734035  8021 tablet_replica.cc:333] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.735664  8021 raft_consensus.cc:2243] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:37.739168  8021 raft_consensus.cc:2272] T 49752620cbbf430991915a2400f71286 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.742363  8021 tablet_replica.cc:333] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.743434  8021 raft_consensus.cc:2243] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:06:37.745019  8021 raft_consensus.cc:2272] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.748365  8021 tablet_replica.cc:333] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.749294  8021 raft_consensus.cc:2243] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.750422  8021 raft_consensus.cc:2272] T 6cdcabf0b1ec4b319a537ed3662ce253 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.817551  8021 tablet_replica.cc:333] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.818722  8021 raft_consensus.cc:2243] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:37.830674  9825 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:39939, user_credentials={real_user=slave}} blocked reactor thread for 79402us
I20260709 13:06:37.853581  8021 raft_consensus.cc:2272] T 2cfe3483aaf54fbd932e71b72394491c P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.857892  8021 tablet_replica.cc:333] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.858523  8021 raft_consensus.cc:2243] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.859153  8021 raft_consensus.cc:2272] T 55b71b09f6834fbbbd500874ff1e14b5 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.874264  8021 tablet_replica.cc:333] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.879995  8021 raft_consensus.cc:2243] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:06:37.881287  8021 raft_consensus.cc:2272] T c07692445f814d5cab7dd13d622f9b9c P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.884383  8021 tablet_replica.cc:333] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.885382  8021 raft_consensus.cc:2243] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.886281  8021 raft_consensus.cc:2272] T bbfd5d6417134672a62b2757b6d55dff P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.889529  8021 tablet_replica.cc:333] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.891484  8021 raft_consensus.cc:2243] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.892293  8021 raft_consensus.cc:2272] T aaf63fb358d848ad991a929b2e8e0168 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.895166  8021 tablet_replica.cc:333] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.896111  8021 raft_consensus.cc:2243] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.896809  8021 raft_consensus.cc:2272] T 7cfb4241815c471a8b29031ba9674528 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:37.898057  9238 consensus_peers.cc:597] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:37.899792  8021 tablet_replica.cc:333] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.901711  8021 raft_consensus.cc:2243] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.902426  8021 raft_consensus.cc:2272] T 0a06199955d24d13ba9589faa3f11ebe P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:37.902652  9828 consensus_peers.cc:597] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.903342  9828 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.903955  9828 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.903292  9238 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.905944  9238 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:37.906968  8021 tablet_replica.cc:333] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.907977  8021 raft_consensus.cc:2243] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Raft consensus shutting down.
W20260709 13:06:37.908818  9828 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:37.909060  8021 raft_consensus.cc:2272] T 909051b50b2c464ea2b13a15a243817c P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:37.911079  9238 consensus_peers.cc:597] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:37.912220  8021 tablet_replica.cc:333] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.912973  8021 raft_consensus.cc:2243] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.913755  8021 raft_consensus.cc:2272] T 1abbfa6c01a144acb9bc31470b44be7e P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.916320  8021 tablet_replica.cc:333] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.917135  8021 raft_consensus.cc:2243] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:06:37.918301  8021 raft_consensus.cc:2272] T a5204a9a27e54ddcb19d93142d8156f7 P 94504d29c6744f38bb06943479df9466 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:37.919169  9828 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:37.922294  8021 tablet_replica.cc:333] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.923337  8021 raft_consensus.cc:2243] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.924012  8021 raft_consensus.cc:2272] T 9e2655d16bd349309dab52831e66fee8 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.926584  8021 tablet_replica.cc:333] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
W20260709 13:06:37.927803  9828 consensus_peers.cc:597] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:37.928913  8021 raft_consensus.cc:2243] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:37.930339  8021 raft_consensus.cc:2272] T 8a4bd074bf594b89b28b47d7482e8ac1 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.932993  8021 tablet_replica.cc:333] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.933733  8021 raft_consensus.cc:2243] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.938875  8021 raft_consensus.cc:2272] T 1fdaa422821a46b8bc73801d1abfb3c0 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.941859  8021 tablet_replica.cc:333] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.942765  8021 raft_consensus.cc:2243] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:37.943965  8021 raft_consensus.cc:2272] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.946794  8021 tablet_replica.cc:333] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.947624  8021 raft_consensus.cc:2243] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.948293  8021 raft_consensus.cc:2272] T 017b65b967c44e7688e66e7aab235b62 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.950989  8021 tablet_replica.cc:333] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.951781  8021 raft_consensus.cc:2243] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:06:37.952913  8021 raft_consensus.cc:2272] T c351454182cf49938eb92870b1fb297e P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.955448  8021 tablet_replica.cc:333] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.956571  8021 raft_consensus.cc:2243] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Raft consensus shutting down.
W20260709 13:06:37.960356  9828 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:37.960938  9828 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:37.961233  8021 raft_consensus.cc:2272] T c089688bfc69403095596a002ccbf890 P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.964071  8021 tablet_replica.cc:333] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
W20260709 13:06:37.967694  9238 consensus_peers.cc:597] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:37.972951  8021 raft_consensus.cc:2243] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.973764  8021 raft_consensus.cc:2272] T d8f97d26e68b4fbead130c830dbf4b51 P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.977912  8021 tablet_replica.cc:333] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.978870  8021 raft_consensus.cc:2243] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:37.979480  8021 raft_consensus.cc:2272] T e9a6ffaa328644f4bd7f129f8fa22875 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.982331  8021 tablet_replica.cc:333] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.983026  8021 raft_consensus.cc:2243] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:06:37.984125  8021 raft_consensus.cc:2272] T 991b28944986479a8ad096ca92f1a435 P 94504d29c6744f38bb06943479df9466 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:37.986781  8021 tablet_replica.cc:333] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:37.987550  8021 raft_consensus.cc:2243] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:06:37.989430  8021 raft_consensus.cc:2272] T 06c3356087cc45e78fa0f79168b68e8c P 94504d29c6744f38bb06943479df9466 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.003628  8021 tablet_replica.cc:333] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
W20260709 13:06:38.004484  9238 consensus_peers.cc:597] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:38.004585  8021 raft_consensus.cc:2243] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.005497  8021 raft_consensus.cc:2272] T 776f24c8f05046a0a730da5e6e801cb7 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.008520  8021 tablet_replica.cc:333] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.009470  8021 raft_consensus.cc:2243] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:06:38.010620  8021 raft_consensus.cc:2272] T fa5405d9d7b34438b7f5732807fac54a P 94504d29c6744f38bb06943479df9466 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.014024  8021 tablet_replica.cc:333] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.014818  8021 raft_consensus.cc:2243] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.015475  8021 raft_consensus.cc:2272] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.025979  8021 tablet_replica.cc:333] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.026839  8021 raft_consensus.cc:2243] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.027832  8021 raft_consensus.cc:2272] T f84c9fc901364e60b2c414ee3b7a7e65 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.030745  8021 tablet_replica.cc:333] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.031493  8021 raft_consensus.cc:2243] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.032709  8021 raft_consensus.cc:2272] T 7597d602dc674623ae1d5de5fdb5d2ee P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:38.037092  9828 consensus_peers.cc:597] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:38.038725  8021 tablet_replica.cc:333] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.039646  8021 raft_consensus.cc:2243] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.040459  8021 raft_consensus.cc:2272] T 7c3c420051574414bbec6a77f8036b61 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.043220  8021 tablet_replica.cc:333] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.044275  8021 raft_consensus.cc:2243] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.045099  8021 raft_consensus.cc:2272] T 29257e9d0bc4408583c5df86d6e2fd64 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.047850  8021 tablet_replica.cc:333] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.048974  8021 raft_consensus.cc:2243] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.049587  8021 raft_consensus.cc:2272] T 9a3eaf34ab444dca94a52a6fce2a556e P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.052630  8021 tablet_replica.cc:333] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.053422  8021 raft_consensus.cc:2243] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:38.054610  8021 raft_consensus.cc:2272] T a6c1cc32069145ebbbacf127d2516fe8 P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.057442  8021 tablet_replica.cc:333] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.058219  8021 raft_consensus.cc:2243] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:06:38.059309  8021 raft_consensus.cc:2272] T 1c199cae5ff14a478726c8a8b491f8f7 P 94504d29c6744f38bb06943479df9466 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.062638  8021 tablet_replica.cc:333] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.063400  8021 raft_consensus.cc:2243] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:38.065143  8021 raft_consensus.cc:2272] T edfac5554fb144c3ab8a6248c7ea68d8 P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.067962  8021 tablet_replica.cc:333] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.068809  8021 raft_consensus.cc:2243] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.069756  8021 raft_consensus.cc:2272] T ff8d71c72ec64addaf0593eca7fd672d P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.072732  8021 tablet_replica.cc:333] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.073637  8021 raft_consensus.cc:2243] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.074280  8021 raft_consensus.cc:2272] T a35f9598c50046539a7f41287876135d P 94504d29c6744f38bb06943479df9466 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.077652  8021 tablet_replica.cc:333] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.078509  8021 raft_consensus.cc:2243] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.079290  8021 raft_consensus.cc:2272] T dc3a66792b31404ba18eaacb7c54d08b P 94504d29c6744f38bb06943479df9466 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.086483  8021 tablet_replica.cc:333] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.087321  8021 raft_consensus.cc:2243] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.088057  8021 raft_consensus.cc:2272] T 43e874ff0724409b88c5ec377b73a06a P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.091080  8021 tablet_replica.cc:333] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.091835  8021 raft_consensus.cc:2243] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:06:38.092988  8021 raft_consensus.cc:2272] T 23636a3a86df4bdf91dc6a7ffe723527 P 94504d29c6744f38bb06943479df9466 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.095758  8021 tablet_replica.cc:333] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.096629  8021 raft_consensus.cc:2243] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:38.097821  8021 raft_consensus.cc:2272] T a51a0795b52e4843959cd3be3d077713 P 94504d29c6744f38bb06943479df9466 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.100682  8021 tablet_replica.cc:333] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466: stopping tablet replica
W20260709 13:06:38.104166  9828 consensus_peers.cc:597] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:38.105300  8021 raft_consensus.cc:2243] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.106001  8021 raft_consensus.cc:2272] T dfa5110f517a4ffb866953379f5d0c4d P 94504d29c6744f38bb06943479df9466 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:38.116276  9825 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:38.119292  8021 tablet_replica.cc:333] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.120183  8021 raft_consensus.cc:2243] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:38.121320  8021 raft_consensus.cc:2272] T e58cbd03222246299b7e4363d8fcffa6 P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:38.124260  8021 tablet_replica.cc:333] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.124961  8021 raft_consensus.cc:2243] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:38.125595  8021 raft_consensus.cc:2272] T b0917335c1234d5abec1e40c742de6be P 94504d29c6744f38bb06943479df9466 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:38.129305  9235 consensus_peers.cc:597] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:38.134353  8021 tablet_replica.cc:333] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466: stopping tablet replica
I20260709 13:06:38.135267  8021 raft_consensus.cc:2243] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:06:38.136732  8021 raft_consensus.cc:2272] T 98d913b72e5b43e8b8f20fb99d0838c4 P 94504d29c6744f38bb06943479df9466 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:38.160208  9825 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.192207  9825 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.283450  9235 consensus_peers.cc:597] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:38.292948  9235 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:06:38.298832  9235 consensus_peers.cc:597] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.336776  9235 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.350359  9235 consensus_peers.cc:597] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.363310  9235 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.427685  9825 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.436373  9825 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:38.709148  9825 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.714057  9235 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.860530  9235 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.943424  9235 consensus_peers.cc:597] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:38.944573  9825 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:38.954620 15980 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:38.959227 15980 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:38.950516 15977 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:38.967243 15977 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:38.970459 15980 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:38.985687 15979 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.006158 15979 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.008631 15979 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:39.010057 15985 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.010694 15985 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.012557 15989 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.013188 15985 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:39.013166 15989 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.016482 15989 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:39.004691 15977 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:39.020452  9235 leader_election.cc:336] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:39.021507 15980 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.026191 15985 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.027200 15985 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:39.030457 15985 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:39.027204 15980 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.032886 15980 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:39.038511  9825 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:39.051120  9266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:39.051437  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:39.058207  9825 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:39.062493  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
W20260709 13:06:39.051509  9238 leader_election.cc:336] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.068288 15980 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.070073  9238 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.070221 15980 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:39.072170  9828 leader_election.cc:336] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.072649 15980 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:39.069501 15973 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.073591 15973 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.076562 15973 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:39.077134 15985 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:06:39.076386  9235 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:39.084379  9235 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:39.084551 15973 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
W20260709 13:06:39.085089  9235 consensus_peers.cc:597] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:39.085582  9235 consensus_peers.cc:597] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:39.085181 15973 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.087354 15973 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:39.089026  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:06:39.089787  9238 leader_election.cc:336] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.090256  9826 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.091096 15977 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:39.091822 15977 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:39.092712 15980 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.093345 15980 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:39.095886 15980 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:39.100263  9825 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:39.102967  9825 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:39.106057  9238 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 92919b70789041c8a0de0b3525154fd1, 94504d29c6744f38bb06943479df9466
W20260709 13:06:39.107489  9825 consensus_peers.cc:597] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:39.108049  9235 leader_election.cc:336] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:39.091820  9828 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.110071  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.110687 15980 raft_consensus.cc:2749] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:06:39.114423 15988 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:39.114619  9828 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.117626  9862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:39.118268  9862 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 18.
W20260709 13:06:39.123723  9825 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:39.126991  9235 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:39.129446  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:39.131249  9264 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 18.
W20260709 13:06:39.146450  9235 consensus_peers.cc:597] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:39.154065 15980 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.154747 15980 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:39.156980 15980 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:39.162549  9828 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:39.167640  9238 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.170533  9238 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:39.169983  9235 leader_election.cc:336] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:39.171774 15980 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:39.172806  9238 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:39.178509  9825 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:39.176566  9238 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.180985 15980 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:39.181983  9238 leader_election.cc:336] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.182974  9238 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.182440  9828 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.183930 15988 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:06:39.184218  9828 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.184669  9828 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.185401 15988 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:39.188225 15977 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.188740 15977 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.190842 15977 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:39.191931 15980 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:06:39.199910 15977 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.200527 15977 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.205402 15977 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:39.207615  9825 leader_election.cc:336] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:39.220227  9238 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.224959  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.228973 15980 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:39.228171  9828 leader_election.cc:336] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:39.251482  9235 consensus_peers.cc:597] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:39.254576  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:39.265676  9235 consensus_peers.cc:597] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:39.270687 15977 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.271294 15977 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.273561 15977 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:39.279814  9825 leader_election.cc:336] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:39.281622  9828 leader_election.cc:336] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.290006  9826 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:39.297012 15988 raft_consensus.cc:2749] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:39.297729  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.298647 15988 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:39.299546  9235 consensus_peers.cc:597] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:39.314671  9238 leader_election.cc:336] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.316684  9238 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.319169 15980 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:39.322654  9828 leader_election.cc:336] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.328862  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.329730 15988 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:39.338054  9825 consensus_peers.cc:597] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:39.355967 15985 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.356747 15985 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.358915 15985 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:39.361666  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:39.362370  9855 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 11.
I20260709 13:06:39.367995  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:39.374333 15985 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:06:39.374748 15985 raft_consensus.cc:493] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.375123 15985 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:39.379119 15988 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.379709 15988 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.381809 15988 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:39.382017 15977 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.382542 15977 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.386336 15977 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:39.389004  9825 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:39.390059  8021 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 9219)
I20260709 13:06:39.398903 15977 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.399461 15977 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.400699 15980 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.401204 15980 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.401600 15977 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:39.403137 15980 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:39.411309  9828 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:39.413203  9238 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:43523, user_credentials={real_user=slave}} blocked reactor thread for 45297.1us
I20260709 13:06:39.416015 15977 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.415719  9828 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.418889 15977 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.419404 15988 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:39.420301  9828 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.421023 15977 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:39.425221  9825 leader_election.cc:336] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:39.432914  9825 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:39.434391 15985 raft_consensus.cc:515] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.436740 15985 leader_election.cc:290] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:39.437685  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
W20260709 13:06:39.439308  9235 consensus_peers.cc:597] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:39.442755  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:39.443432  9264 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 20.
W20260709 13:06:39.446532  9238 leader_election.cc:336] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.447257  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:39.449035  9855 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
I20260709 13:06:39.452625  9826 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:06:39.452826  9238 leader_election.cc:336] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:39.448449  9828 leader_election.cc:336] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.453730  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:39.454121 15988 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:39.455021 15979 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:06:39.455363 15979 raft_consensus.cc:493] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.455706 15979 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:39.460520 15985 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.461222 15985 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:39.466118  9825 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:39.466778  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d913b72e5b43e8b8f20fb99d0838c4" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:39.469012  9855 raft_consensus.cc:3060] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:06:39.469983 15979 raft_consensus.cc:515] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.472118 15980 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.472956 15979 leader_election.cc:290] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:39.472711 15980 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.475569 15980 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:39.470137 15985 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:39.480299  9235 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:39.482514  9855 raft_consensus.cc:2468] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 12.
W20260709 13:06:39.467820  9828 leader_election.cc:336] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.485335  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.488611 15979 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:39.489868  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:39.489918  9264 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06c3356087cc45e78fa0f79168b68e8c" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa"
I20260709 13:06:39.490830  9264 raft_consensus.cc:3060] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:06:39.491097 15985 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:06:39.491498  9238 leader_election.cc:304] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:39.491582 15985 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.492324 15985 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:06:39.494056  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:39.494725  9855 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 17.
I20260709 13:06:39.497814  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:39.498521  9855 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:39.499394 15980 raft_consensus.cc:2804] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:06:39.502564 15985 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.504488  9264 raft_consensus.cc:2468] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 21.
I20260709 13:06:39.514398 15985 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:39.506229  9238 leader_election.cc:336] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:39.508317  9828 leader_election.cc:336] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:39.529004  9235 consensus_peers.cc:597] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:39.540655 15980 raft_consensus.cc:697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:39.541888 15980 consensus_queue.cc:237] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.541399  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:39.556690 15980 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:06:39.558367 15980 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Starting leader election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.559012 15980 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:39.564863 15985 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:39.566327 15985 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:39.568308  9826 leader_election.cc:304] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:39.569584 15985 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:06:39.571282 15979 raft_consensus.cc:2804] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:06:39.571864 15979 raft_consensus.cc:697] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 21 LEADER]: Becoming Leader. State: Replica: 92919b70789041c8a0de0b3525154fd1, State: Running, Role: LEADER
I20260709 13:06:39.572958 15979 consensus_queue.cc:237] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.577004 15980 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:39.579206  9235 leader_election.cc:336] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:39.580514  9238 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.582293  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5405d9d7b34438b7f5732807fac54a" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:39.601011  9855 raft_consensus.cc:3060] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:06:39.601208  9238 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.601792  9238 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.603502 15991 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:39.609048  9855 raft_consensus.cc:2468] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 13.
I20260709 13:06:39.613906  9013 catalog_manager.cc:5697] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 11 to 12, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 12 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:39.618346  9238 leader_election.cc:336] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.629256  9238 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466
I20260709 13:06:39.620241 15980 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Requested vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
I20260709 13:06:39.619705  9019 catalog_manager.cc:5697] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 reported cstate change: term changed from 20 to 21, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 92919b70789041c8a0de0b3525154fd1 (127.7.213.68). New cstate: current_term: 21 leader_uuid: "92919b70789041c8a0de0b3525154fd1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:06:39.635776 15985 raft_consensus.cc:2804] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:06:39.636400 15985 raft_consensus.cc:697] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:39.637427 15985 consensus_queue.cc:237] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:39.651836  9855 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edfac5554fb144c3ab8a6248c7ea68d8" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1"
I20260709 13:06:39.652526  9855 raft_consensus.cc:3060] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:06:39.660527  9855 raft_consensus.cc:2468] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 18.
W20260709 13:06:39.686933  9238 leader_election.cc:336] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.687515  9238 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:39.688341 15980 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:39.698096  9238 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 
I20260709 13:06:39.698961 15980 raft_consensus.cc:2804] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:06:39.699457 15980 raft_consensus.cc:697] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 18 LEADER]: Becoming Leader. State: Replica: 1efabcb1befc400aadf101e8276fc1aa, State: Running, Role: LEADER
I20260709 13:06:39.700505 15980 consensus_queue.cc:237] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:39.714828  9238 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:39.715858  9019 catalog_manager.cc:5697] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 12 to 13, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 13 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:39.739537  9019 catalog_manager.cc:5697] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa reported cstate change: term changed from 17 to 18, leader changed from 94504d29c6744f38bb06943479df9466 (127.7.213.66) to 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67). New cstate: current_term: 18 leader_uuid: "1efabcb1befc400aadf101e8276fc1aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:40.008880  9828 consensus_peers.cc:597] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:40.031792  9238 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.067721  9238 consensus_peers.cc:597] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:40.068394  9238 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.069087  9238 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:40.080380  9862 raft_consensus.cc:1275] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:06:40.101445  9828 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.103989  9238 consensus_peers.cc:597] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.104517  9238 consensus_peers.cc:597] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:40.106002 15980 consensus_queue.cc:1048] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:06:40.109961  9828 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:40.122849  9266 raft_consensus.cc:1275] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Refusing update from remote peer 92919b70789041c8a0de0b3525154fd1: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:06:40.127573 15988 consensus_queue.cc:1048] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:06:40.141481  9238 consensus_peers.cc:597] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.193499  9828 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.200675  9238 consensus_peers.cc:597] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:40.209157  9828 consensus_peers.cc:597] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:40.209220  9862 raft_consensus.cc:1275] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:06:40.241147 15991 consensus_queue.cc:1048] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:06:40.238619  9828 consensus_peers.cc:597] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:40.261911  9862 raft_consensus.cc:1275] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Refusing update from remote peer 1efabcb1befc400aadf101e8276fc1aa: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:06:40.275092 15991 consensus_queue.cc:1048] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:06:40.340992  9238 consensus_peers.cc:597] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.353783  9828 consensus_peers.cc:597] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.362432  9828 consensus_peers.cc:597] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.370431  9238 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.383683  9828 consensus_peers.cc:597] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.394387  8021 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 9219)
W20260709 13:06:40.421289  9238 consensus_peers.cc:597] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.439245  9828 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.444041  9238 consensus_peers.cc:597] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.461978  9828 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.519649  9238 consensus_peers.cc:597] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.563452  9825 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:40.572655  9828 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.617556  9235 consensus_peers.cc:597] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:40.637424  9825 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:40.639264  9828 consensus_peers.cc:597] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.663470  9238 consensus_peers.cc:597] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.676671  9828 consensus_peers.cc:597] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:06:40.689267 15988 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:40.690096 15988 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:40.692672 15988 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:40.694115  9825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111) [suppressed 593 similar messages]
W20260709 13:06:40.697604  9825 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:40.705408  9235 consensus_peers.cc:597] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:40.708442  9828 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:40.709061  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:40.709987 15988 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:40.735271  9825 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:40.786054  9235 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:40.800222  9828 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:34417, user_credentials={real_user=slave}} blocked reactor thread for 43840.6us
I20260709 13:06:40.801734 15991 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:40.805750 15991 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:40.816215  9235 consensus_peers.cc:597] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:40.827370  9825 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:40.818775 15991 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 92919b70789041c8a0de0b3525154fd1 (127.7.213.68:43523)
W20260709 13:06:40.833050  9238 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:40.834254  9235 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:40.835124  9859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4bd074bf594b89b28b47d7482e8ac1" candidate_uuid: "1efabcb1befc400aadf101e8276fc1aa" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "92919b70789041c8a0de0b3525154fd1" is_pre_election: true
I20260709 13:06:40.839876  9859 raft_consensus.cc:2468] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1efabcb1befc400aadf101e8276fc1aa in term 18.
I20260709 13:06:40.841208  9235 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:40.854831 15980 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:40.869163 15980 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:40.870332 15980 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:40.872428 15980 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:40.882634  9235 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:06:40.901480  9828 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.916909  9828 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.917833  9828 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:40.906787  9238 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:40.918951  9235 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:40.925377  9235 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:40.933223 15980 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:40.934268  9235 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:40.943661  9825 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:40.955639  9235 consensus_peers.cc:597] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:06:40.964596 15980 raft_consensus.cc:493] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:40.965268 15980 raft_consensus.cc:515] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:40.967411 15980 leader_election.cc:290] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:40.985713  9238 leader_election.cc:336] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:40.991290  9235 leader_election.cc:336] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:40.992058  9235 leader_election.cc:304] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:40.992992 15980 raft_consensus.cc:2749] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:40.997859 16011 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:40.998622 16011 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:41.001004 16011 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:41.015795  9828 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:41.017536  9825 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:41.020404  9825 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.029093 16011 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:06:41.031122 16011 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:41.031817 16011 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:41.033910 16011 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:41.046703  9828 leader_election.cc:336] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:41.067343  9825 leader_election.cc:336] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:41.073163  9825 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.076380 15977 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:41.109865 15980 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:41.110634 15980 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:41.117772  9235 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:41.112654 15980 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:41.150298  9238 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:41.164659  9238 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.167218 15980 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:41.189771  9825 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:41.194823  9235 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:06:41.237092 15980 raft_consensus.cc:493] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:41.237860 15980 raft_consensus.cc:515] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:41.240479 15977 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:41.241266 15977 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:41.244699 15980 leader_election.cc:290] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:41.247922  9825 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:41.248024 15977 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:41.250691  9263 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "991b28944986479a8ad096ca92f1a435" candidate_uuid: "92919b70789041c8a0de0b3525154fd1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1efabcb1befc400aadf101e8276fc1aa" is_pre_election: true
I20260709 13:06:41.251441  9263 raft_consensus.cc:2468] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92919b70789041c8a0de0b3525154fd1 in term 13.
W20260709 13:06:41.257903  9238 leader_election.cc:336] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:41.261648 16011 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:41.262306 16011 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:41.264473 16011 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:41.266396  9235 leader_election.cc:336] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:41.267355  9235 leader_election.cc:304] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.269219 15980 raft_consensus.cc:493] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:41.272080 15980 raft_consensus.cc:515] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:41.273418  9825 leader_election.cc:336] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:41.275712 15980 leader_election.cc:290] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939)
I20260709 13:06:41.277460 15980 raft_consensus.cc:2749] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:41.288610  9828 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:41.296375  9828 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa, 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.298864 15977 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:41.297657  9238 leader_election.cc:336] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:41.299990  9235 leader_election.cc:336] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:41.302568  9828 leader_election.cc:336] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:41.303187  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.300935  9235 leader_election.cc:304] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.306813 15990 raft_consensus.cc:2749] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:41.310696 15977 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:41.386042  9825 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:41.394927  8021 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 9219)
W20260709 13:06:41.396996  9235 consensus_peers.cc:597] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:41.399655  9235 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:06:41.411893 15990 raft_consensus.cc:493] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:41.412695 15990 raft_consensus.cc:515] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:41.415228 15990 leader_election.cc:290] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:41.420667  9238 leader_election.cc:336] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:41.421779  9235 leader_election.cc:336] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:41.422621  9235 leader_election.cc:304] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.423611 15990 raft_consensus.cc:2749] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:41.464335  9825 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:41.477568  9235 consensus_peers.cc:597] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:41.480209  9235 consensus_peers.cc:597] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:41.613842  9825 consensus_peers.cc:597] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:41.640480  9825 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:41.641346  9825 consensus_peers.cc:597] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:41.653508  9235 consensus_peers.cc:597] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:06:41.665542 16011 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:41.666397 16011 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:41.675640 16011 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:41.700911 15977 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:41.701686 15977 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:41.704023 15977 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:41.717043  9235 consensus_peers.cc:597] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:41.724639  9828 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:41.725698  9825 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:41.726197  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.727953 16011 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:06:41.735014  9235 consensus_peers.cc:597] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:41.735836  9825 leader_election.cc:336] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:41.737974  9828 leader_election.cc:336] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:41.738444  9828 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:41.740375 15977 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:41.810001  9825 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:41.912405  9825 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:41.926977  9235 consensus_peers.cc:597] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:42.015514  9235 consensus_peers.cc:597] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:42.236156  9235 consensus_peers.cc:597] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:42.328287  9238 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.328775  9238 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.338449  9238 consensus_peers.cc:597] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:42.395545  8021 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 9219)
W20260709 13:06:42.406424  9828 consensus_peers.cc:597] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.412945  9828 consensus_peers.cc:597] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:42.424501  9238 consensus_peers.cc:597] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:42.442233  9238 consensus_peers.cc:597] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.625808  9828 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.643435  9828 consensus_peers.cc:597] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.704944  9238 consensus_peers.cc:597] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.733855  9828 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.760109  9238 consensus_peers.cc:597] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.760908  9238 consensus_peers.cc:597] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:06:42.761541  9238 consensus_peers.cc:597] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.773864  9828 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.791761  9828 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.793709  9828 consensus_peers.cc:597] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.796831  9238 consensus_peers.cc:597] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.819077  9828 consensus_peers.cc:597] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.850044  9828 consensus_peers.cc:597] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.872790  9238 consensus_peers.cc:597] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.897706  9238 consensus_peers.cc:597] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.924309  9238 consensus_peers.cc:597] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:42.990927  9235 consensus_peers.cc:597] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:42.997277  9828 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:43.098081  9825 consensus_peers.cc:597] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:43.110476  9828 consensus_peers.cc:597] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:43.127528  9235 consensus_peers.cc:597] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:06:43.144124 15990 raft_consensus.cc:493] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:43.144645 15990 raft_consensus.cc:515] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
I20260709 13:06:43.150082 15990 leader_election.cc:290] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:43.162534  9238 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:43.163306 16014 raft_consensus.cc:493] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:43.163911 16014 raft_consensus.cc:515] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } }
W20260709 13:06:43.165830  9235 leader_election.cc:336] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:43.165962 16014 leader_election.cc:290] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:43.170509  9238 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:43.166240  9235 leader_election.cc:304] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:43.171659 16014 raft_consensus.cc:2749] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:06:43.176522  9235 leader_election.cc:336] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:43.176977  9235 leader_election.cc:304] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1efabcb1befc400aadf101e8276fc1aa; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:43.178038 16014 raft_consensus.cc:2749] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:43.185760  9828 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:43.213261  8021 tablet_server.cc:196] TabletServer@127.7.213.66:0 shutdown complete.
W20260709 13:06:43.213441  9235 consensus_peers.cc:597] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:06:43.235319  9825 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:43.237982  9828 consensus_peers.cc:597] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:43.259181 16013 raft_consensus.cc:493] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:43.259956 16013 raft_consensus.cc:515] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:43.262640 16013 leader_election.cc:290] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:43.268218  9825 leader_election.cc:336] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:43.273463  9828 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:43.273983  9828 leader_election.cc:336] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:43.274402  9828 leader_election.cc:304] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:43.275245 16013 raft_consensus.cc:2749] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:06:43.278012  9238 consensus_peers.cc:597] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:43.298713  9828 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:06:43.332568  9825 consensus_peers.cc:597] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:43.339458  9235 consensus_peers.cc:597] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:43.358108  9235 consensus_peers.cc:597] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:06:43.373003  8021 tablet_server.cc:179] TabletServer@127.7.213.67:0 shutting down...
W20260709 13:06:43.403641  9235 consensus_peers.cc:597] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:43.416162  9826 consensus_peers.cc:597] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:43.450136  9825 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:43.460268  9825 consensus_peers.cc:597] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:43.461906  9828 consensus_peers.cc:597] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:43.466032  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:43.467118  8021 tablet_replica.cc:333] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.467906  8021 raft_consensus.cc:2243] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Raft consensus shutting down.
W20260709 13:06:43.469007  9826 consensus_peers.cc:597] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:43.469143  8021 raft_consensus.cc:2272] T c0f89ac8f7b142239d6103bd6ed96037 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.471616  8021 tablet_replica.cc:333] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.472312  8021 raft_consensus.cc:2243] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.473234  8021 raft_consensus.cc:2272] T 6cdcabf0b1ec4b319a537ed3662ce253 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.475622  8021 tablet_replica.cc:333] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.476464  8021 raft_consensus.cc:2243] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.477524  8021 raft_consensus.cc:2272] T 709565930ea044f08322d94e29d83adf P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.479660  8021 tablet_replica.cc:333] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.480289  8021 raft_consensus.cc:2243] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.480804  8021 raft_consensus.cc:2272] T aa32d6e0bb7b44ee9f664126e3ecdb72 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.483440  8021 tablet_replica.cc:333] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.484338  8021 raft_consensus.cc:2243] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.484938  8021 raft_consensus.cc:2272] T 49752620cbbf430991915a2400f71286 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.487730  8021 tablet_replica.cc:333] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
W20260709 13:06:43.488095  9826 consensus_peers.cc:597] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:43.488634  8021 raft_consensus.cc:2243] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.489902  8021 raft_consensus.cc:2272] T 2cfe3483aaf54fbd932e71b72394491c P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:43.492702  9826 consensus_peers.cc:597] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:43.492889  8021 tablet_replica.cc:333] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.493909  8021 raft_consensus.cc:2243] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.495195  8021 raft_consensus.cc:2272] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.499022  8021 tablet_replica.cc:333] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.499645  8021 raft_consensus.cc:2243] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.500301  8021 raft_consensus.cc:2272] T aaf63fb358d848ad991a929b2e8e0168 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.503046  8021 tablet_replica.cc:333] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.503834  8021 raft_consensus.cc:2243] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.504511  8021 raft_consensus.cc:2272] T 55cb8917093e4273bb3af2d4d36815cf P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.507016  8021 tablet_replica.cc:333] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.507617  8021 raft_consensus.cc:2243] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.508370  8021 raft_consensus.cc:2272] T c07692445f814d5cab7dd13d622f9b9c P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.510634  8021 tablet_replica.cc:333] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.511166  8021 raft_consensus.cc:2243] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.511773  8021 raft_consensus.cc:2272] T bbfd5d6417134672a62b2757b6d55dff P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.513921  8021 tablet_replica.cc:333] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.514515  8021 raft_consensus.cc:2243] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.515446  8021 raft_consensus.cc:2272] T 00ff82ce7b7342a4b052db43182f4a99 P 1efabcb1befc400aadf101e8276fc1aa [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.517993  8021 tablet_replica.cc:333] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.518760  8021 raft_consensus.cc:2243] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.519383  8021 raft_consensus.cc:2272] T 1abbfa6c01a144acb9bc31470b44be7e P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.521860  8021 tablet_replica.cc:333] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.522425  8021 raft_consensus.cc:2243] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.523340  8021 raft_consensus.cc:2272] T 470326df6cae476c90888533e504e6ec P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.525333  8021 tablet_replica.cc:333] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.525957  8021 raft_consensus.cc:2243] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.526667  8021 raft_consensus.cc:2272] T 0a06199955d24d13ba9589faa3f11ebe P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.529670  8021 tablet_replica.cc:333] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.530619  8021 raft_consensus.cc:2243] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.531387  8021 raft_consensus.cc:2272] T a5204a9a27e54ddcb19d93142d8156f7 P 1efabcb1befc400aadf101e8276fc1aa [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.534039  8021 tablet_replica.cc:333] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.534848  8021 raft_consensus.cc:2243] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.535898  8021 raft_consensus.cc:2272] T 9e2655d16bd349309dab52831e66fee8 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.538259  8021 tablet_replica.cc:333] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.538834  8021 raft_consensus.cc:2243] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.539526  8021 raft_consensus.cc:2272] T 8a4bd074bf594b89b28b47d7482e8ac1 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.541869  8021 tablet_replica.cc:333] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.542579  8021 raft_consensus.cc:2243] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.543851  8021 raft_consensus.cc:2272] T bc89bbd2f2934bbeaf33619d3d23adfc P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.546316  8021 tablet_replica.cc:333] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.546890  8021 raft_consensus.cc:2243] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.548156  8021 raft_consensus.cc:2272] T 2b7da62e1fea4b79963197feee655d26 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.550884  8021 tablet_replica.cc:333] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.551466  8021 raft_consensus.cc:2243] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.552268  8021 raft_consensus.cc:2272] T 1fdaa422821a46b8bc73801d1abfb3c0 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.555023  8021 tablet_replica.cc:333] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.555874  8021 raft_consensus.cc:2243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.557255  8021 raft_consensus.cc:2272] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 1efabcb1befc400aadf101e8276fc1aa [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.560075  8021 tablet_replica.cc:333] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.560772  8021 raft_consensus.cc:2243] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.561494  8021 raft_consensus.cc:2272] T 7cfb4241815c471a8b29031ba9674528 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.563586  8021 tablet_replica.cc:333] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.564185  8021 raft_consensus.cc:2243] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.565229  8021 raft_consensus.cc:2272] T a08146adfeb9417ca7067de38da555b0 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.567279  8021 tablet_replica.cc:333] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.568027  8021 raft_consensus.cc:2243] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.568683  8021 raft_consensus.cc:2272] T c351454182cf49938eb92870b1fb297e P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.572309  8021 tablet_replica.cc:333] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.572899  8021 raft_consensus.cc:2243] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.573774  8021 raft_consensus.cc:2272] T c089688bfc69403095596a002ccbf890 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.575889  8021 tablet_replica.cc:333] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.576503  8021 raft_consensus.cc:2243] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.577481  8021 raft_consensus.cc:2272] T d8f97d26e68b4fbead130c830dbf4b51 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.580077  8021 tablet_replica.cc:333] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.580806  8021 raft_consensus.cc:2243] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.581537  8021 raft_consensus.cc:2272] T e9a6ffaa328644f4bd7f129f8fa22875 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.583873  8021 tablet_replica.cc:333] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.584424  8021 raft_consensus.cc:2243] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.585266  8021 raft_consensus.cc:2272] T 991b28944986479a8ad096ca92f1a435 P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.587291  8021 tablet_replica.cc:333] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.587898  8021 raft_consensus.cc:2243] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.588502  8021 raft_consensus.cc:2272] T 9daa912f9e08438caf063c0c83e09fe5 P 1efabcb1befc400aadf101e8276fc1aa [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.590337  8021 tablet_replica.cc:333] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.591029  8021 raft_consensus.cc:2243] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.592052  8021 raft_consensus.cc:2272] T fa5405d9d7b34438b7f5732807fac54a P 1efabcb1befc400aadf101e8276fc1aa [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.594137  8021 tablet_replica.cc:333] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.594679  8021 raft_consensus.cc:2243] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.595175  8021 raft_consensus.cc:2272] T 06c3356087cc45e78fa0f79168b68e8c P 1efabcb1befc400aadf101e8276fc1aa [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.597146  8021 tablet_replica.cc:333] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.597659  8021 raft_consensus.cc:2243] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.598778  8021 raft_consensus.cc:2272] T 776f24c8f05046a0a730da5e6e801cb7 P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.601588  8021 tablet_replica.cc:333] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
W20260709 13:06:43.602330  9826 consensus_peers.cc:597] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:43.602391  8021 raft_consensus.cc:2243] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.603434  8021 raft_consensus.cc:2272] T ca0b93b18a7f4e6398018531838527d9 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.606752  8021 tablet_replica.cc:333] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.607349  8021 raft_consensus.cc:2243] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.609145  8021 raft_consensus.cc:2272] T 98d913b72e5b43e8b8f20fb99d0838c4 P 1efabcb1befc400aadf101e8276fc1aa [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.610942  8021 tablet_replica.cc:333] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.611497  8021 raft_consensus.cc:2243] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.612654  8021 raft_consensus.cc:2272] T 90c9bcfd54594ec58c5e8e1be6c556d8 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.614312  8021 tablet_replica.cc:333] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.614821  8021 raft_consensus.cc:2243] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.615697  8021 raft_consensus.cc:2272] T f84c9fc901364e60b2c414ee3b7a7e65 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.617621  8021 tablet_replica.cc:333] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.618144  8021 raft_consensus.cc:2243] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.618618  8021 raft_consensus.cc:2272] T bd9140502bd3469eb208753dea0f3159 P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.620878  8021 tablet_replica.cc:333] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.621500  8021 raft_consensus.cc:2243] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.622627  8021 raft_consensus.cc:2272] T 7c3c420051574414bbec6a77f8036b61 P 1efabcb1befc400aadf101e8276fc1aa [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.624899  8021 tablet_replica.cc:333] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.625469  8021 raft_consensus.cc:2243] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.626278  8021 raft_consensus.cc:2272] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 1efabcb1befc400aadf101e8276fc1aa [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.628350  8021 tablet_replica.cc:333] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.628862  8021 raft_consensus.cc:2243] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.630090  8021 raft_consensus.cc:2272] T edfac5554fb144c3ab8a6248c7ea68d8 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.632264  8021 tablet_replica.cc:333] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.632930  8021 raft_consensus.cc:2243] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.633806  8021 raft_consensus.cc:2272] T dc3a66792b31404ba18eaacb7c54d08b P 1efabcb1befc400aadf101e8276fc1aa [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.635747  8021 tablet_replica.cc:333] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.636408  8021 raft_consensus.cc:2243] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:43.637634  8021 raft_consensus.cc:2272] T dfa5110f517a4ffb866953379f5d0c4d P 1efabcb1befc400aadf101e8276fc1aa [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.640009  8021 tablet_replica.cc:333] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa: stopping tablet replica
I20260709 13:06:43.640556  8021 raft_consensus.cc:2243] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:43.641100  8021 raft_consensus.cc:2272] T e58cbd03222246299b7e4363d8fcffa6 P 1efabcb1befc400aadf101e8276fc1aa [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:43.676973 15977 raft_consensus.cc:493] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:43.677521 15977 raft_consensus.cc:515] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:43.679435 15977 leader_election.cc:290] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:43.684036  9826 consensus_peers.cc:597] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:43.694133  9825 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:43.699029  9828 leader_election.cc:336] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:43.699532  9828 leader_election.cc:304] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:43.700856 16013 raft_consensus.cc:2749] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:06:43.701747  9826 consensus_peers.cc:597] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:43.732096  9825 consensus_peers.cc:597] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:43.752038  9826 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:43.788764  9826 consensus_peers.cc:597] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:43.796469  9826 consensus_peers.cc:597] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:43.849162 15977 raft_consensus.cc:493] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:43.849628 15977 raft_consensus.cc:515] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:43.851404 15977 leader_election.cc:290] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:43.855907  9826 consensus_peers.cc:597] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:06:43.857754  9825 leader_election.cc:336] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:43.863026  9828 leader_election.cc:336] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:43.863481  9828 leader_election.cc:304] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:43.864395 16013 raft_consensus.cc:2749] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:43.881450  9825 consensus_peers.cc:597] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:43.934798  9826 consensus_peers.cc:597] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 -> Peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Couldn't send request to peer 1efabcb1befc400aadf101e8276fc1aa. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:43.938242 16013 raft_consensus.cc:493] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:43.938939 16013 raft_consensus.cc:515] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:43.946146  9828 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:43.949354  9825 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:43.949631 16013 leader_election.cc:290] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:43.950019  9825 leader_election.cc:304] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
W20260709 13:06:43.950718  9826 leader_election.cc:336] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:43.951107 16013 raft_consensus.cc:2749] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:06:44.001073 16018 raft_consensus.cc:493] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:44.001748 16018 raft_consensus.cc:515] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:44.013373  9825 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:44.015353  9828 leader_election.cc:336] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:44.015802  9828 leader_election.cc:304] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.016853 15977 raft_consensus.cc:2749] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:06:44.015822 16018 leader_election.cc:290] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
I20260709 13:06:44.327858 16018 raft_consensus.cc:493] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:44.328449 16018 raft_consensus.cc:515] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } attrs { replace: true } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.330948 16018 leader_election.cc:290] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:44.379525 15977 raft_consensus.cc:493] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:44.380105 15977 raft_consensus.cc:515] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
W20260709 13:06:44.382539  9826 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
W20260709 13:06:44.395480  9828 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:44.397926  9828 leader_election.cc:304] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:44.400708 16013 raft_consensus.cc:2749] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:44.404377 15977 leader_election.cc:290] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:44.417474  9825 consensus_peers.cc:597] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:44.418149  9825 leader_election.cc:336] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:44.418998  9825 consensus_peers.cc:597] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:44.419601  9825 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:44.421851  9828 leader_election.cc:336] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:44.424012  9825 leader_election.cc:336] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:44.424381  9825 leader_election.cc:304] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.425168 16022 raft_consensus.cc:2749] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:06:44.425384  9825 consensus_peers.cc:597] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:06:44.430068  9825 consensus_peers.cc:597] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:06:44.441663  9825 consensus_peers.cc:597] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 -> Peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Couldn't send request to peer ce51b61da22f40d4aa0204a6b1028f28. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:06:44.520655 16022 raft_consensus.cc:493] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.521241 16022 raft_consensus.cc:515] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.523099 16022 leader_election.cc:290] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.527390  9826 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
W20260709 13:06:44.529161  9825 leader_election.cc:336] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:44.529524  9825 leader_election.cc:304] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.530125 15977 raft_consensus.cc:2749] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:44.577127 15977 raft_consensus.cc:493] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.577706 15977 raft_consensus.cc:515] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.579635 15977 leader_election.cc:290] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.583858  9826 leader_election.cc:336] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
W20260709 13:06:44.585439  9825 leader_election.cc:336] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:44.585808  9825 leader_election.cc:304] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.586354 15977 raft_consensus.cc:2749] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:44.643923 15977 raft_consensus.cc:493] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.644462 15977 raft_consensus.cc:515] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.646370 15977 leader_election.cc:290] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.649904  9826 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
W20260709 13:06:44.650458  9825 leader_election.cc:336] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:44.650830  9825 leader_election.cc:304] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.651392 15977 raft_consensus.cc:2749] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:06:44.656491 15977 raft_consensus.cc:493] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.656955 15977 raft_consensus.cc:515] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.658763 15977 leader_election.cc:290] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.662036  9825 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:44.662550  9826 leader_election.cc:336] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:44.662969  9826 leader_election.cc:304] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.663260 15977 raft_consensus.cc:493] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.663831 16022 raft_consensus.cc:2749] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:06:44.663815 15977 raft_consensus.cc:515] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.666185 15977 leader_election.cc:290] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.669862  9825 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:44.670399  9826 leader_election.cc:336] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:44.670806  9826 leader_election.cc:304] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.671412 15977 raft_consensus.cc:2749] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:06:44.684969 15977 raft_consensus.cc:493] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.685483 15977 raft_consensus.cc:515] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.687345 16022 raft_consensus.cc:493] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.687903 16022 raft_consensus.cc:515] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.688357 15977 leader_election.cc:290] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:44.689837 16022 leader_election.cc:290] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.691370  9828 leader_election.cc:336] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:44.693748  9826 leader_election.cc:336] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
W20260709 13:06:44.694164  9825 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:44.694186  9826 leader_election.cc:304] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:44.695006 16022 raft_consensus.cc:2749] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:06:44.695168  9826 leader_election.cc:336] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:44.695601  9826 leader_election.cc:304] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.696215 16022 raft_consensus.cc:2749] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.705054 16022 raft_consensus.cc:493] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.705497 16022 raft_consensus.cc:515] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.707432 16022 leader_election.cc:290] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.711465  9825 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:44.712004  9826 leader_election.cc:336] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:44.712335  9826 leader_election.cc:304] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.712941 16022 raft_consensus.cc:2749] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:06:44.724898 16022 raft_consensus.cc:493] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.725461 16022 raft_consensus.cc:515] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.727520 16022 leader_election.cc:290] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.731134  9825 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
W20260709 13:06:44.731616  9826 leader_election.cc:336] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:44.732033  9826 leader_election.cc:304] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.732683 16022 raft_consensus.cc:2749] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.758423  8021 tablet_server.cc:196] TabletServer@127.7.213.67:0 shutdown complete.
I20260709 13:06:44.777670 16022 raft_consensus.cc:493] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.778244 16022 raft_consensus.cc:515] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.780325 16022 leader_election.cc:290] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.784817  9828 leader_election.cc:336] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:44.785318  9826 leader_election.cc:336] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:44.785775  9826 leader_election.cc:304] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:44.786517 16022 raft_consensus.cc:2749] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.837045 16021 raft_consensus.cc:493] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.837795 16021 raft_consensus.cc:515] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.840562 16021 leader_election.cc:290] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:44.841817 16021 raft_consensus.cc:493] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.842367 16021 raft_consensus.cc:515] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.845055 16021 leader_election.cc:290] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.847924  9828 leader_election.cc:336] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:44.849318  9826 leader_election.cc:336] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:44.851552  9826 leader_election.cc:304] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
W20260709 13:06:44.852515  9825 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:44.853165 16021 raft_consensus.cc:2749] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:06:44.855222  9826 leader_election.cc:336] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:44.855729  9826 leader_election.cc:304] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.856451 16021 raft_consensus.cc:2749] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.861028 16021 raft_consensus.cc:493] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 94504d29c6744f38bb06943479df9466)
I20260709 13:06:44.861552 16021 raft_consensus.cc:515] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.862687  8021 tablet_server.cc:179] TabletServer@127.7.213.68:0 shutting down...
I20260709 13:06:44.865116 16021 leader_election.cc:290] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417)
W20260709 13:06:44.868880  9828 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
W20260709 13:06:44.869495  9825 leader_election.cc:336] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:44.869921  9825 leader_election.cc:304] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 94504d29c6744f38bb06943479df9466, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.870604 16021 raft_consensus.cc:2749] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:06:44.882200 16021 raft_consensus.cc:493] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.882807 16021 raft_consensus.cc:515] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.885419 16021 leader_election.cc:290] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
W20260709 13:06:44.892395  9826 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
W20260709 13:06:44.892905  9828 leader_election.cc:336] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:44.893425  9828 leader_election.cc:304] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:44.894227 16021 raft_consensus.cc:2749] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:06:44.914256 16021 raft_consensus.cc:493] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
W20260709 13:06:44.915963  9828 consensus_peers.cc:597] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:06:44.916867 16021 raft_consensus.cc:515] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ce51b61da22f40d4aa0204a6b1028f28" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 39939 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.918375  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:06:44.919252  8021 tablet_replica.cc:333] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.920055 16021 leader_election.cc:290] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:44.920198  8021 raft_consensus.cc:2243] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.921186  8021 raft_consensus.cc:2272] T 1c199cae5ff14a478726c8a8b491f8f7 P 92919b70789041c8a0de0b3525154fd1 [term 10 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:44.923615  9825 leader_election.cc:336] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer ce51b61da22f40d4aa0204a6b1028f28 (127.7.213.65:39939): Network error: Client connection negotiation failed: client connection to 127.7.213.65:39939: connect: Connection refused (error 111)
I20260709 13:06:44.924088  8021 tablet_replica.cc:333] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
W20260709 13:06:44.924602  9826 leader_election.cc:336] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:44.924926  8021 raft_consensus.cc:2243] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.924943  9826 leader_election.cc:304] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, ce51b61da22f40d4aa0204a6b1028f28
I20260709 13:06:44.925820  8021 raft_consensus.cc:2272] T 991b28944986479a8ad096ca92f1a435 P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.925760 16021 raft_consensus.cc:2749] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:06:44.928475  8021 tablet_replica.cc:333] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.929284  8021 raft_consensus.cc:2243] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.930402  8021 raft_consensus.cc:2272] T 017b65b967c44e7688e66e7aab235b62 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.932348  8021 tablet_replica.cc:333] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.932883  8021 raft_consensus.cc:2243] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.933337  8021 raft_consensus.cc:2272] T 8a4bd074bf594b89b28b47d7482e8ac1 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.935156  8021 tablet_replica.cc:333] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.935685  8021 raft_consensus.cc:2243] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.936235  8021 raft_consensus.cc:2272] T bc89bbd2f2934bbeaf33619d3d23adfc P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.938196  8021 tablet_replica.cc:333] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.938748  8021 raft_consensus.cc:2243] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.939777  8021 raft_consensus.cc:2272] T 55cb8917093e4273bb3af2d4d36815cf P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.942503  8021 tablet_replica.cc:333] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
W20260709 13:06:44.942662  9828 consensus_peers.cc:597] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 -> Peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Couldn't send request to peer 94504d29c6744f38bb06943479df9466. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:06:44.943295  8021 raft_consensus.cc:2243] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.944324  8021 raft_consensus.cc:2272] T 6cdcabf0b1ec4b319a537ed3662ce253 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.946295  8021 tablet_replica.cc:333] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.946832  8021 raft_consensus.cc:2243] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.947283  8021 raft_consensus.cc:2272] T a51a0795b52e4843959cd3be3d077713 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.949087  8021 tablet_replica.cc:333] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.949666  8021 raft_consensus.cc:2243] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.950147  8021 raft_consensus.cc:2272] T 709565930ea044f08322d94e29d83adf P 92919b70789041c8a0de0b3525154fd1 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.951910  8021 tablet_replica.cc:333] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.952445  8021 raft_consensus.cc:2243] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.952900  8021 raft_consensus.cc:2272] T 98d913b72e5b43e8b8f20fb99d0838c4 P 92919b70789041c8a0de0b3525154fd1 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.954632  8021 tablet_replica.cc:333] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.955147  8021 raft_consensus.cc:2243] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.956038  8021 raft_consensus.cc:2272] T b0917335c1234d5abec1e40c742de6be P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.957835  8021 tablet_replica.cc:333] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.958384  8021 raft_consensus.cc:2243] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.959365  8021 raft_consensus.cc:2272] T 06c3356087cc45e78fa0f79168b68e8c P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.961462  8021 tablet_replica.cc:333] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.962059  8021 raft_consensus.cc:2243] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.963092  8021 raft_consensus.cc:2272] T ca0b93b18a7f4e6398018531838527d9 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.965240  8021 tablet_replica.cc:333] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.965792  8021 raft_consensus.cc:2243] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.966202  8021 raft_consensus.cc:2272] T 685fa3ec557f4f3db02ec6ccce1d6f3e P 92919b70789041c8a0de0b3525154fd1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.968124  8021 tablet_replica.cc:333] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.968652  8021 raft_consensus.cc:2243] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.969400  8021 raft_consensus.cc:2272] T 1fdaa422821a46b8bc73801d1abfb3c0 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.971141  8021 tablet_replica.cc:333] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.971731  8021 raft_consensus.cc:2243] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.972265  8021 raft_consensus.cc:2272] T db3bbfae79ea4f4cb8b7e3079bdb3908 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.974028  8021 tablet_replica.cc:333] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.974572  8021 raft_consensus.cc:2243] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.975404  8021 raft_consensus.cc:2272] T 1abbfa6c01a144acb9bc31470b44be7e P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.977192  8021 tablet_replica.cc:333] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.977737  8021 raft_consensus.cc:2243] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.978250  8021 raft_consensus.cc:2272] T 9e2655d16bd349309dab52831e66fee8 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.979943  8021 tablet_replica.cc:333] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.980476  8021 raft_consensus.cc:2243] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.981271  8021 raft_consensus.cc:2272] T 0a06199955d24d13ba9589faa3f11ebe P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.983860  8021 tablet_replica.cc:333] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.984601  8021 raft_consensus.cc:2243] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.985600  8021 raft_consensus.cc:2272] T 7cfb4241815c471a8b29031ba9674528 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.987838  8021 tablet_replica.cc:333] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.988538  8021 raft_consensus.cc:2243] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.989223  8021 raft_consensus.cc:2272] T 43e874ff0724409b88c5ec377b73a06a P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.990819  8021 tablet_replica.cc:333] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.991333  8021 raft_consensus.cc:2243] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.991816  8021 raft_consensus.cc:2272] T 23636a3a86df4bdf91dc6a7ffe723527 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.993528  8021 tablet_replica.cc:333] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.994050  8021 raft_consensus.cc:2243] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:44.994501  8021 raft_consensus.cc:2272] T 909051b50b2c464ea2b13a15a243817c P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.996305  8021 tablet_replica.cc:333] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:44.996803  8021 raft_consensus.cc:2243] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:06:44.997664  8021 raft_consensus.cc:2272] T a35f9598c50046539a7f41287876135d P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:44.997786 16022 raft_consensus.cc:493] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1efabcb1befc400aadf101e8276fc1aa)
I20260709 13:06:44.998361 16022 raft_consensus.cc:515] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "94504d29c6744f38bb06943479df9466" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 34417 } } peers { permanent_uuid: "1efabcb1befc400aadf101e8276fc1aa" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 38807 } } peers { permanent_uuid: "92919b70789041c8a0de0b3525154fd1" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 43523 } attrs { promote: false } }
I20260709 13:06:44.999966  8021 tablet_replica.cc:333] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.000291 16022 leader_election.cc:290] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417), 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807)
I20260709 13:06:45.000737  8021 raft_consensus.cc:2243] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.001780  8021 raft_consensus.cc:2272] T 470326df6cae476c90888533e504e6ec P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:45.003188  9828 leader_election.cc:336] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 94504d29c6744f38bb06943479df9466 (127.7.213.66:34417): Network error: Client connection negotiation failed: client connection to 127.7.213.66:34417: connect: Connection refused (error 111)
I20260709 13:06:45.004194  8021 tablet_replica.cc:333] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
W20260709 13:06:45.004629  9826 leader_election.cc:336] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1efabcb1befc400aadf101e8276fc1aa (127.7.213.67:38807): Network error: Client connection negotiation failed: client connection to 127.7.213.67:38807: connect: Connection refused (error 111)
I20260709 13:06:45.004870  8021 raft_consensus.cc:2243] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.004984  9826 leader_election.cc:304] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92919b70789041c8a0de0b3525154fd1; no voters: 1efabcb1befc400aadf101e8276fc1aa, 94504d29c6744f38bb06943479df9466
I20260709 13:06:45.005564  8021 raft_consensus.cc:2272] T a08146adfeb9417ca7067de38da555b0 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.005681 16022 raft_consensus.cc:2749] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:06:45.007472  8021 tablet_replica.cc:333] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.008116  8021 raft_consensus.cc:2243] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.008929  8021 raft_consensus.cc:2272] T bd9140502bd3469eb208753dea0f3159 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.010797  8021 tablet_replica.cc:333] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.011298  8021 raft_consensus.cc:2243] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.012086  8021 raft_consensus.cc:2272] T f84c9fc901364e60b2c414ee3b7a7e65 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.014161  8021 tablet_replica.cc:333] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.014853  8021 raft_consensus.cc:2243] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.015609  8021 raft_consensus.cc:2272] T ff8d71c72ec64addaf0593eca7fd672d P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.017288  8021 tablet_replica.cc:333] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.017746  8021 raft_consensus.cc:2243] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.018160  8021 raft_consensus.cc:2272] T edfac5554fb144c3ab8a6248c7ea68d8 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.019829  8021 tablet_replica.cc:333] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.020270  8021 raft_consensus.cc:2243] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.020608  8021 raft_consensus.cc:2272] T fa5405d9d7b34438b7f5732807fac54a P 92919b70789041c8a0de0b3525154fd1 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.022145  8021 tablet_replica.cc:333] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.022578  8021 raft_consensus.cc:2243] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.023437  8021 raft_consensus.cc:2272] T 9daa912f9e08438caf063c0c83e09fe5 P 92919b70789041c8a0de0b3525154fd1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.025116  8021 tablet_replica.cc:333] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.025549  8021 raft_consensus.cc:2243] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.026221  8021 raft_consensus.cc:2272] T bbfd5d6417134672a62b2757b6d55dff P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.027944  8021 tablet_replica.cc:333] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.028373  8021 raft_consensus.cc:2243] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.028858  8021 raft_consensus.cc:2272] T 9cbe8a4b1d764261adeeb8d882fa37e5 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.030560  8021 tablet_replica.cc:333] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.030982  8021 raft_consensus.cc:2243] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.031733  8021 raft_consensus.cc:2272] T 7597d602dc674623ae1d5de5fdb5d2ee P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.033480  8021 tablet_replica.cc:333] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.033928  8021 raft_consensus.cc:2243] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.034389  8021 raft_consensus.cc:2272] T c0f89ac8f7b142239d6103bd6ed96037 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.036170  8021 tablet_replica.cc:333] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.036891  8021 raft_consensus.cc:2243] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.037308  8021 raft_consensus.cc:2272] T 00ff82ce7b7342a4b052db43182f4a99 P 92919b70789041c8a0de0b3525154fd1 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.039644  8021 tablet_replica.cc:333] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.040289  8021 raft_consensus.cc:2243] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.040642  8021 raft_consensus.cc:2272] T 2b7da62e1fea4b79963197feee655d26 P 92919b70789041c8a0de0b3525154fd1 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.042207  8021 tablet_replica.cc:333] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.042604  8021 raft_consensus.cc:2243] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.043424  8021 raft_consensus.cc:2272] T 29257e9d0bc4408583c5df86d6e2fd64 P 92919b70789041c8a0de0b3525154fd1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.045198  8021 tablet_replica.cc:333] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.045634  8021 raft_consensus.cc:2243] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.046324  8021 raft_consensus.cc:2272] T aaf63fb358d848ad991a929b2e8e0168 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.047966  8021 tablet_replica.cc:333] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.048475  8021 raft_consensus.cc:2243] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.049221  8021 raft_consensus.cc:2272] T 55b71b09f6834fbbbd500874ff1e14b5 P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.050891  8021 tablet_replica.cc:333] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.051352  8021 raft_consensus.cc:2243] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.051928  8021 raft_consensus.cc:2272] T 8d9ec1fba08a49aeb3e1a8a52e798291 P 92919b70789041c8a0de0b3525154fd1 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.053579  8021 tablet_replica.cc:333] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.054016  8021 raft_consensus.cc:2243] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.054762  8021 raft_consensus.cc:2272] T e9a6ffaa328644f4bd7f129f8fa22875 P 92919b70789041c8a0de0b3525154fd1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.056500  8021 tablet_replica.cc:333] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.056939  8021 raft_consensus.cc:2243] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:06:45.057736  8021 raft_consensus.cc:2272] T 9a3eaf34ab444dca94a52a6fce2a556e P 92919b70789041c8a0de0b3525154fd1 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:45.059450  8021 tablet_replica.cc:333] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1: stopping tablet replica
I20260709 13:06:45.059942  8021 raft_consensus.cc:2243] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:06:45.060340  8021 raft_consensus.cc:2272] T a6c1cc32069145ebbbacf127d2516fe8 P 92919b70789041c8a0de0b3525154fd1 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:06:45.339488  9883 debug-util.cc:398] Leaking SignalData structure 0x7b080039fae0 after lost signal to thread 8024
W20260709 13:06:45.340487  9883 debug-util.cc:398] Leaking SignalData structure 0x7b08003c4dc0 after lost signal to thread 9055
W20260709 13:06:45.341537  9883 debug-util.cc:398] Leaking SignalData structure 0x7b0800264a00 after lost signal to thread 9887
I20260709 13:06:46.254022  8021 tablet_server.cc:196] TabletServer@127.7.213.68:0 shutdown complete.
I20260709 13:06:46.316401  8021 master.cc:562] Master@127.7.213.126:45141 shutting down...
I20260709 13:06:46.349032  8021 raft_consensus.cc:2243] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:06:46.349666  8021 raft_consensus.cc:2272] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:06:46.350018  8021 tablet_replica.cc:333] T 00000000000000000000000000000000 P cdc5c504a10a4da1812e571d14ae3abd: stopping tablet replica
I20260709 13:06:46.473498  8021 master.cc:584] Master@127.7.213.126:45141 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (224721 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:06:46.578131  8021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.213.126:46449
I20260709 13:06:46.579187  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:46.585357 16024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:46.585417 16023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:46.586501 16026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:46.587838  8021 server_base.cc:1061] running on GCE node
I20260709 13:06:46.590446  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:46.590623  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:46.590736  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602406590725 us; error 0 us; skew 500 ppm
I20260709 13:06:46.591180  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:46.593973  8021 webserver.cc:533] Webserver started at http://127.7.213.126:37275/ using document root <none> and password file <none>
I20260709 13:06:46.594415  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:46.594568  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:46.594777  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:46.596004  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/master-0-root/instance:
uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0"
format_stamp: "Formatted at 2026-07-09 13:06:46 on dist-test-slave-9g29"
I20260709 13:06:46.600487  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:06:46.603848 16031 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:46.604610  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:06:46.604900  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/master-0-root
uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0"
format_stamp: "Formatted at 2026-07-09 13:06:46 on dist-test-slave-9g29"
I20260709 13:06:46.605216  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:46.618268  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:46.619441  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:46.658317  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.126:46449
I20260709 13:06:46.658422 16082 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.126:46449 every 8 connection(s)
I20260709 13:06:46.662362 16083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:46.672441 16083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0: Bootstrap starting.
I20260709 13:06:46.677162 16083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:46.681423 16083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0: No bootstrap required, opened a new log
I20260709 13:06:46.683673 16083 raft_consensus.cc:359] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0" member_type: VOTER }
I20260709 13:06:46.684123 16083 raft_consensus.cc:385] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:46.684329 16083 raft_consensus.cc:740] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33af400ee6fd4c1da4b7aa37a0c7b2b0, State: Initialized, Role: FOLLOWER
I20260709 13:06:46.684999 16083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0" member_type: VOTER }
I20260709 13:06:46.685431 16083 raft_consensus.cc:399] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:06:46.685671 16083 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:06:46.685927 16083 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:46.690158 16083 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0" member_type: VOTER }
I20260709 13:06:46.690671 16083 leader_election.cc:304] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33af400ee6fd4c1da4b7aa37a0c7b2b0; no voters: 
I20260709 13:06:46.691844 16083 leader_election.cc:290] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:06:46.692391 16086 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:46.693922 16086 raft_consensus.cc:697] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 1 LEADER]: Becoming Leader. State: Replica: 33af400ee6fd4c1da4b7aa37a0c7b2b0, State: Running, Role: LEADER
I20260709 13:06:46.694684 16086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0" member_type: VOTER }
I20260709 13:06:46.694877 16083 sys_catalog.cc:565] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:06:46.699795 16088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33af400ee6fd4c1da4b7aa37a0c7b2b0. Latest consensus state: current_term: 1 leader_uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0" member_type: VOTER } }
I20260709 13:06:46.699790 16087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33af400ee6fd4c1da4b7aa37a0c7b2b0" member_type: VOTER } }
I20260709 13:06:46.700572 16088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:46.700680 16087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [sys.catalog]: This master's current role is: LEADER
I20260709 13:06:46.703442 16093 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:06:46.709126 16093 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:06:46.712584  8021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:06:46.719156 16093 catalog_manager.cc:1383] Generated new cluster ID: 0f51091ee7034186a32f4e017d58d2f6
I20260709 13:06:46.719565 16093 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:06:46.754279 16093 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:06:46.755601 16093 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:06:46.771154 16093 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0: Generated new TSK 0
I20260709 13:06:46.771878 16093 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:06:46.781885  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:46.788376 16104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:46.789723 16105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:46.791190 16107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:46.792441  8021 server_base.cc:1061] running on GCE node
I20260709 13:06:46.796809  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:46.797070  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:46.797263  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602406797242 us; error 0 us; skew 500 ppm
I20260709 13:06:46.797889  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:46.800401  8021 webserver.cc:533] Webserver started at http://127.7.213.65:38325/ using document root <none> and password file <none>
I20260709 13:06:46.800922  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:46.801129  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:46.801422  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:46.802634  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/instance:
uuid: "aa8eb07867e7420b80a7e615d570509d"
format_stamp: "Formatted at 2026-07-09 13:06:46 on dist-test-slave-9g29"
I20260709 13:06:46.807941  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 13:06:46.811450 16112 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:46.812270  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:46.812621  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "aa8eb07867e7420b80a7e615d570509d"
format_stamp: "Formatted at 2026-07-09 13:06:46 on dist-test-slave-9g29"
I20260709 13:06:46.812904  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:46.831112  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:46.832453  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:46.834023  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:46.836283  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:46.836532  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:46.836773  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:46.836957  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:46.876794  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:40593
I20260709 13:06:46.876895 16174 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:40593 every 8 connection(s)
I20260709 13:06:46.881752  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:46.889163 16179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:46.890894 16180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:46.892993 16182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:46.893831  8021 server_base.cc:1061] running on GCE node
I20260709 13:06:46.895975  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:46.896162  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:46.896355  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602406896325 us; error 0 us; skew 500 ppm
I20260709 13:06:46.896901  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:46.899268  8021 webserver.cc:533] Webserver started at http://127.7.213.66:34289/ using document root <none> and password file <none>
I20260709 13:06:46.899943  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:46.900182  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:46.900453  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:46.901911  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/instance:
uuid: "3ce6da6d71124256be0679451014f776"
format_stamp: "Formatted at 2026-07-09 13:06:46 on dist-test-slave-9g29"
I20260709 13:06:46.903174 16175 heartbeater.cc:344] Connected to a master server at 127.7.213.126:46449
I20260709 13:06:46.903581 16175 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:46.904320 16175 heartbeater.cc:507] Master 127.7.213.126:46449 requested a full tablet report, sending...
I20260709 13:06:46.906805 16048 ts_manager.cc:194] Registered new tserver with Master: aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:46.906941  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:06:46.908644 16048 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36558
I20260709 13:06:46.911209 16187 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:46.912178  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:06:46.912478  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-1-root
uuid: "3ce6da6d71124256be0679451014f776"
format_stamp: "Formatted at 2026-07-09 13:06:46 on dist-test-slave-9g29"
I20260709 13:06:46.912772  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:46.934355  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:46.935582  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:46.937063  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:46.939265  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:46.939482  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:46.939720  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:46.939944  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:06:46.981287  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.66:43495
I20260709 13:06:46.981400 16249 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.66:43495 every 8 connection(s)
I20260709 13:06:46.986222  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:46.993487 16253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:46.995484 16254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:46.999051  8021 server_base.cc:1061] running on GCE node
I20260709 13:06:46.999887 16250 heartbeater.cc:344] Connected to a master server at 127.7.213.126:46449
W20260709 13:06:47.000124 16256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:47.000355 16250 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:47.001255  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:06:47.001344 16250 heartbeater.cc:507] Master 127.7.213.126:46449 requested a full tablet report, sending...
W20260709 13:06:47.001581  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:47.001813  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602407001797 us; error 0 us; skew 500 ppm
I20260709 13:06:47.002547  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:47.003911 16048 ts_manager.cc:194] Registered new tserver with Master: 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:47.005355  8021 webserver.cc:533] Webserver started at http://127.7.213.67:39739/ using document root <none> and password file <none>
I20260709 13:06:47.005868 16048 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36574
I20260709 13:06:47.006071  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:47.006361  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:47.006740  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:47.007980  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/instance:
uuid: "115c2d35eda349178144f8365481e89c"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-9g29"
I20260709 13:06:47.012439  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:06:47.016532 16261 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.017455  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:06:47.017728  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-2-root
uuid: "115c2d35eda349178144f8365481e89c"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-9g29"
I20260709 13:06:47.017978  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:47.029749  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:47.030895  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:47.032404  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:47.034744  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:47.034956  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.035239  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:47.035393  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.075670  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.67:36993
I20260709 13:06:47.075843 16323 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.67:36993 every 8 connection(s)
I20260709 13:06:47.080506  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:06:47.086853 16327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.088673 16328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:06:47.090792 16330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:06:47.091806  8021 server_base.cc:1061] running on GCE node
I20260709 13:06:47.092730  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:06:47.092971  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:06:47.093063 16324 heartbeater.cc:344] Connected to a master server at 127.7.213.126:46449
I20260709 13:06:47.093199  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602407093179 us; error 0 us; skew 500 ppm
I20260709 13:06:47.093536 16324 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:47.093994  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:06:47.094449 16324 heartbeater.cc:507] Master 127.7.213.126:46449 requested a full tablet report, sending...
I20260709 13:06:47.096844  8021 webserver.cc:533] Webserver started at http://127.7.213.68:44649/ using document root <none> and password file <none>
I20260709 13:06:47.097050 16048 ts_manager.cc:194] Registered new tserver with Master: 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:47.097455  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:06:47.097724  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:06:47.097987  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:06:47.098708 16048 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36584
I20260709 13:06:47.099149  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-3-root/instance:
uuid: "c8106927ed954e1c800869427b6c7964"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-9g29"
I20260709 13:06:47.103679  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:06:47.107230 16335 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.108107  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:06:47.108402  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-3-root
uuid: "c8106927ed954e1c800869427b6c7964"
format_stamp: "Formatted at 2026-07-09 13:06:47 on dist-test-slave-9g29"
I20260709 13:06:47.108695  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:06:47.118168  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:06:47.119488  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:06:47.121048  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:06:47.123224  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:06:47.123445  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:06:47.123708  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:06:47.123930  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:06:47.163517  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.68:36791
I20260709 13:06:47.163630 16397 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.68:36791 every 8 connection(s)
I20260709 13:06:47.177203 16398 heartbeater.cc:344] Connected to a master server at 127.7.213.126:46449
I20260709 13:06:47.177755 16398 heartbeater.cc:461] Registering TS with master...
I20260709 13:06:47.178685 16398 heartbeater.cc:507] Master 127.7.213.126:46449 requested a full tablet report, sending...
I20260709 13:06:47.180604 16048 ts_manager.cc:194] Registered new tserver with Master: c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:47.181463  8021 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.014918614s
I20260709 13:06:47.181933 16048 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36588
I20260709 13:06:47.207633 16048 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36604:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:06:47.446955 16363 tablet_service.cc:1511] Processing CreateTablet for tablet 55b5e6c6090448ab831f8afeef43db0d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:47.448696 16363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b5e6c6090448ab831f8afeef43db0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.452960 16362 tablet_service.cc:1511] Processing CreateTablet for tablet bd508336a7c844cc84117b7ef3a565ba (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:47.454483 16362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd508336a7c844cc84117b7ef3a565ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.456447 16289 tablet_service.cc:1511] Processing CreateTablet for tablet 55b5e6c6090448ab831f8afeef43db0d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:47.456604 16215 tablet_service.cc:1511] Processing CreateTablet for tablet 829bf55d32c344af8ae546eb15903faa (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:47.457075 16361 tablet_service.cc:1511] Processing CreateTablet for tablet 7e218598a3114f679b2483143a2f4692 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:47.458005 16289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b5e6c6090448ab831f8afeef43db0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.458160 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 829bf55d32c344af8ae546eb15903faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.458557 16285 tablet_service.cc:1511] Processing CreateTablet for tablet 44f529392f494e6d83be5434d71ed0ee (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:47.459390 16360 tablet_service.cc:1511] Processing CreateTablet for tablet f603bf0f99d0428194f0998267e8713d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:47.460234 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44f529392f494e6d83be5434d71ed0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.460438 16214 tablet_service.cc:1511] Processing CreateTablet for tablet bd508336a7c844cc84117b7ef3a565ba (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:47.461985 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd508336a7c844cc84117b7ef3a565ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.464105 16360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f603bf0f99d0428194f0998267e8713d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.465106 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 7e218598a3114f679b2483143a2f4692 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:47.465855 16361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e218598a3114f679b2483143a2f4692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.466626 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e218598a3114f679b2483143a2f4692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.466414 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 7e218598a3114f679b2483143a2f4692 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:06:47.467914 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e218598a3114f679b2483143a2f4692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.469116 16288 tablet_service.cc:1511] Processing CreateTablet for tablet 829bf55d32c344af8ae546eb15903faa (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:47.470676 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 829bf55d32c344af8ae546eb15903faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.470798 16211 tablet_service.cc:1511] Processing CreateTablet for tablet f603bf0f99d0428194f0998267e8713d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:47.469098 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 8f5d69137b6e468bb06eb4212b198f84 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:47.465114 16286 tablet_service.cc:1511] Processing CreateTablet for tablet 8f5d69137b6e468bb06eb4212b198f84 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:47.473717 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f5d69137b6e468bb06eb4212b198f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.475484 16359 tablet_service.cc:1511] Processing CreateTablet for tablet 44f529392f494e6d83be5434d71ed0ee (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:47.476989 16359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44f529392f494e6d83be5434d71ed0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.480118 16211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f603bf0f99d0428194f0998267e8713d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.484105 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f5d69137b6e468bb06eb4212b198f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.507282 16140 tablet_service.cc:1511] Processing CreateTablet for tablet 55b5e6c6090448ab831f8afeef43db0d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:06:47.508811 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b5e6c6090448ab831f8afeef43db0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.510586 16210 tablet_service.cc:1511] Processing CreateTablet for tablet ceba3482809245aa87c4e743269df937 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:47.512172 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceba3482809245aa87c4e743269df937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.511984 16139 tablet_service.cc:1511] Processing CreateTablet for tablet 829bf55d32c344af8ae546eb15903faa (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:06:47.513494 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 829bf55d32c344af8ae546eb15903faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.517016 16137 tablet_service.cc:1511] Processing CreateTablet for tablet 8f5d69137b6e468bb06eb4212b198f84 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:06:47.518505 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f5d69137b6e468bb06eb4212b198f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.520303 16136 tablet_service.cc:1511] Processing CreateTablet for tablet f603bf0f99d0428194f0998267e8713d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:06:47.521914 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f603bf0f99d0428194f0998267e8713d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.525635 16138 tablet_service.cc:1511] Processing CreateTablet for tablet bd508336a7c844cc84117b7ef3a565ba (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:06:47.526980 16284 tablet_service.cc:1511] Processing CreateTablet for tablet ceba3482809245aa87c4e743269df937 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:47.537264 16138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd508336a7c844cc84117b7ef3a565ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.544930 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceba3482809245aa87c4e743269df937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.544701 16135 tablet_service.cc:1511] Processing CreateTablet for tablet 44f529392f494e6d83be5434d71ed0ee (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:06:47.543160 16358 tablet_service.cc:1511] Processing CreateTablet for tablet ceba3482809245aa87c4e743269df937 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:06:47.546617 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44f529392f494e6d83be5434d71ed0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.547812 16358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceba3482809245aa87c4e743269df937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.560966 16209 tablet_service.cc:1511] Processing CreateTablet for tablet f228d4a700de40ada51433b8043bf6a7 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:47.562604 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f228d4a700de40ada51433b8043bf6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.577339 16416 tablet_bootstrap.cc:492] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:47.578689 16134 tablet_service.cc:1511] Processing CreateTablet for tablet f228d4a700de40ada51433b8043bf6a7 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:47.580492 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f228d4a700de40ada51433b8043bf6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.585997 16288 tablet_service.cc:1511] Processing CreateTablet for tablet f228d4a700de40ada51433b8043bf6a7 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:06:47.587476 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f228d4a700de40ada51433b8043bf6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.588743 16357 tablet_service.cc:1511] Processing CreateTablet for tablet 2c751e8904a94d8286347a543fbbc90c (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:47.589847 16133 tablet_service.cc:1511] Processing CreateTablet for tablet 2c751e8904a94d8286347a543fbbc90c (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:47.591364 16133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c751e8904a94d8286347a543fbbc90c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.590301 16357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c751e8904a94d8286347a543fbbc90c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.613514 16356 tablet_service.cc:1511] Processing CreateTablet for tablet 361bd03c04594782911afd84f865c9f5 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:47.614971 16356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361bd03c04594782911afd84f865c9f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.618115 16208 tablet_service.cc:1511] Processing CreateTablet for tablet b0d1691a17794035a9fae371cbabb4be (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:47.622257 16355 tablet_service.cc:1511] Processing CreateTablet for tablet b0d1691a17794035a9fae371cbabb4be (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:47.619642 16208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d1691a17794035a9fae371cbabb4be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.633315 16213 tablet_service.cc:1511] Processing CreateTablet for tablet b90ea5d39bbf4d79a4a44959a8f40359 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:47.639791 16285 tablet_service.cc:1511] Processing CreateTablet for tablet 2128e5983b684efeacc628400afa861f (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:47.641256 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2128e5983b684efeacc628400afa861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.641446 16132 tablet_service.cc:1511] Processing CreateTablet for tablet 361bd03c04594782911afd84f865c9f5 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:47.642855 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361bd03c04594782911afd84f865c9f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.644554 16354 tablet_service.cc:1511] Processing CreateTablet for tablet b90ea5d39bbf4d79a4a44959a8f40359 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:47.644783 16289 tablet_service.cc:1511] Processing CreateTablet for tablet 4e9a1ca7eb1348438944c99fcc3eb8e9 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:47.646318 16289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9a1ca7eb1348438944c99fcc3eb8e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.644950 16131 tablet_service.cc:1511] Processing CreateTablet for tablet 2128e5983b684efeacc628400afa861f (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:47.625473 16207 tablet_service.cc:1511] Processing CreateTablet for tablet 2128e5983b684efeacc628400afa861f (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:06:47.648627 16131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2128e5983b684efeacc628400afa861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.649362 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2128e5983b684efeacc628400afa861f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.652242 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 361bd03c04594782911afd84f865c9f5 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:06:47.653637 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361bd03c04594782911afd84f865c9f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.625705 16355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d1691a17794035a9fae371cbabb4be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.658846 16360 tablet_service.cc:1511] Processing CreateTablet for tablet 4e9a1ca7eb1348438944c99fcc3eb8e9 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:47.660907 16354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b90ea5d39bbf4d79a4a44959a8f40359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.610970 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 2c751e8904a94d8286347a543fbbc90c (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:06:47.684767 16415 tablet_bootstrap.cc:492] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:47.700997 16360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9a1ca7eb1348438944c99fcc3eb8e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.693643 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b90ea5d39bbf4d79a4a44959a8f40359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.714459 16215 tablet_service.cc:1511] Processing CreateTablet for tablet 31991d68fa8642189423a50030c4e7d5 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:47.712787 16416 tablet_bootstrap.cc:654] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.715777 16140 tablet_service.cc:1511] Processing CreateTablet for tablet 4e9a1ca7eb1348438944c99fcc3eb8e9 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:06:47.717279 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9a1ca7eb1348438944c99fcc3eb8e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.717085 16134 tablet_service.cc:1511] Processing CreateTablet for tablet b90ea5d39bbf4d79a4a44959a8f40359 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:06:47.730352 16283 tablet_service.cc:1511] Processing CreateTablet for tablet 31991d68fa8642189423a50030c4e7d5 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:47.684605 16288 tablet_service.cc:1511] Processing CreateTablet for tablet b0d1691a17794035a9fae371cbabb4be (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:06:47.750571 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c751e8904a94d8286347a543fbbc90c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.752269 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31991d68fa8642189423a50030c4e7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.755985 16283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31991d68fa8642189423a50030c4e7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.757165 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d1691a17794035a9fae371cbabb4be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.779242 16361 tablet_service.cc:1511] Processing CreateTablet for tablet 31991d68fa8642189423a50030c4e7d5 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:06:47.780702 16361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31991d68fa8642189423a50030c4e7d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.789520 16415 tablet_bootstrap.cc:654] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.798386 16417 tablet_bootstrap.cc:492] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:47.798646 16286 tablet_service.cc:1511] Processing CreateTablet for tablet c14505e666fc47d29cfb0c9bc323384e (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:47.800192 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c14505e666fc47d29cfb0c9bc323384e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.822721 16284 tablet_service.cc:1511] Processing CreateTablet for tablet 3699132ec3bb4f8baa0204a17bda51f9 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:47.824095 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3699132ec3bb4f8baa0204a17bda51f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.826959 16418 tablet_bootstrap.cc:492] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:47.830377 16363 tablet_service.cc:1511] Processing CreateTablet for tablet 3699132ec3bb4f8baa0204a17bda51f9 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:47.847173 16363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3699132ec3bb4f8baa0204a17bda51f9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:47.839128 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:47.841192 16416 tablet_bootstrap.cc:492] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:47.852368 16416 ts_tablet_manager.cc:1403] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.276s	user 0.016s	sys 0.026s
I20260709 13:06:47.854974 16356 tablet_service.cc:1511] Processing CreateTablet for tablet 47e0028829c94c99a91ca4cc4d629a4a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:47.856374 16356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e0028829c94c99a91ca4cc4d629a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.856678 16358 tablet_service.cc:1511] Processing CreateTablet for tablet a6563b9b2e954646b0ce2535138676d0 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:47.857986 16358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6563b9b2e954646b0ce2535138676d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.864743 16362 tablet_service.cc:1511] Processing CreateTablet for tablet 8a58bd7b2451404a8b70547dee94049a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:47.866139 16362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a58bd7b2451404a8b70547dee94049a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.855510 16416 raft_consensus.cc:359] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:47.868394 16416 raft_consensus.cc:385] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.868794 16416 raft_consensus.cc:740] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.869493 16416 consensus_queue.cc:260] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:47.872325 16140 tablet_service.cc:1511] Processing CreateTablet for tablet c14505e666fc47d29cfb0c9bc323384e (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:47.873617 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c14505e666fc47d29cfb0c9bc323384e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.877595 16417 tablet_bootstrap.cc:654] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.884287 16355 tablet_service.cc:1511] Processing CreateTablet for tablet 8b5ad93fb43d4467ac4dc1f70ccb0978 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:47.885679 16355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5ad93fb43d4467ac4dc1f70ccb0978. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.886086 16356 tablet_service.cc:1511] Processing CreateTablet for tablet 3c337868c6fc4e079f20a204e451ab66 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:47.887315 16356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c337868c6fc4e079f20a204e451ab66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.888729 16354 tablet_service.cc:1511] Processing CreateTablet for tablet ffb9eaed164d46ccac187f1878f24e4a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:47.889402 16360 tablet_service.cc:1511] Processing CreateTablet for tablet a042ac05958049c38414fa65d0c39414 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:47.889967 16354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffb9eaed164d46ccac187f1878f24e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.892736 16418 tablet_bootstrap.cc:654] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:47.895778 16360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a042ac05958049c38414fa65d0c39414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.923214 16416 ts_tablet_manager.cc:1434] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.070s	user 0.012s	sys 0.005s
I20260709 13:06:47.927800 16415 tablet_bootstrap.cc:492] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:47.928432 16415 ts_tablet_manager.cc:1403] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.244s	user 0.020s	sys 0.011s
I20260709 13:06:47.928928 16359 tablet_service.cc:1511] Processing CreateTablet for tablet af66e3a8024e47fe925b7afad68d41c2 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:47.930281 16359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af66e3a8024e47fe925b7afad68d41c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.931644 16415 raft_consensus.cc:359] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:47.933638 16415 raft_consensus.cc:385] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:47.917438 16132 tablet_service.cc:1511] Processing CreateTablet for tablet a042ac05958049c38414fa65d0c39414 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:47.933269 16417 tablet_bootstrap.cc:492] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:47.938140 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a042ac05958049c38414fa65d0c39414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:47.938779 16357 tablet_service.cc:1511] Processing CreateTablet for tablet e652cb454b674e399c2ac22c22c05506 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:47.935477 16415 raft_consensus.cc:740] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:47.944851 16417 ts_tablet_manager.cc:1403] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.146s	user 0.014s	sys 0.000s
I20260709 13:06:47.971073 16362 tablet_service.cc:1511] Processing CreateTablet for tablet c0f16aeeb02645aea6c44a180eac4f87 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:47.980717 16175 heartbeater.cc:499] Master 127.7.213.126:46449 was elected leader, sending a full tablet report...
I20260709 13:06:47.981153 16415 consensus_queue.cc:260] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:47.982340 16357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e652cb454b674e399c2ac22c22c05506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.005775 16355 tablet_service.cc:1511] Processing CreateTablet for tablet b521a909fd464b72bc2742ff3157658b (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:48.008335 16363 tablet_service.cc:1511] Processing CreateTablet for tablet abb840ca52d04206a82103fb4eb79ccf (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:48.010277 16135 tablet_service.cc:1511] Processing CreateTablet for tablet 3699132ec3bb4f8baa0204a17bda51f9 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:06:48.010618 16361 tablet_service.cc:1511] Processing CreateTablet for tablet 18645784203b4ce1bbe3302513d910c1 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:48.011840 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3699132ec3bb4f8baa0204a17bda51f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.031113 16136 tablet_service.cc:1511] Processing CreateTablet for tablet c66825803ddc46e8849c7718d3cb56ea (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:48.032739 16362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f16aeeb02645aea6c44a180eac4f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.038944 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c66825803ddc46e8849c7718d3cb56ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.043571 16417 raft_consensus.cc:359] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:48.047888 16356 tablet_service.cc:1511] Processing CreateTablet for tablet 4cec4e308d104a9f9049fa0c2263b58d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:48.048588 16417 raft_consensus.cc:385] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.048923 16417 raft_consensus.cc:740] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.049801 16417 consensus_queue.cc:260] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:48.054646 16285 tablet_service.cc:1511] Processing CreateTablet for tablet a6563b9b2e954646b0ce2535138676d0 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:48.056769 16250 heartbeater.cc:499] Master 127.7.213.126:46449 was elected leader, sending a full tablet report...
I20260709 13:06:48.059021 16361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18645784203b4ce1bbe3302513d910c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.059845 16357 tablet_service.cc:1511] Processing CreateTablet for tablet 7973b570c8554e048bd5a9d02fd13a46 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:48.061275 16357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7973b570c8554e048bd5a9d02fd13a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.061187 16287 tablet_service.cc:1511] Processing CreateTablet for tablet e652cb454b674e399c2ac22c22c05506 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:48.062649 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e652cb454b674e399c2ac22c22c05506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.064677 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6563b9b2e954646b0ce2535138676d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.066699 16324 heartbeater.cc:499] Master 127.7.213.126:46449 was elected leader, sending a full tablet report...
I20260709 13:06:48.069108 16210 tablet_service.cc:1511] Processing CreateTablet for tablet c14505e666fc47d29cfb0c9bc323384e (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:06:48.070427 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c14505e666fc47d29cfb0c9bc323384e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.072705 16214 tablet_service.cc:1511] Processing CreateTablet for tablet 8a58bd7b2451404a8b70547dee94049a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:48.073329 16281 tablet_service.cc:1511] Processing CreateTablet for tablet ffb9eaed164d46ccac187f1878f24e4a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:48.073975 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a58bd7b2451404a8b70547dee94049a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.074620 16281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffb9eaed164d46ccac187f1878f24e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.076774 16418 tablet_bootstrap.cc:492] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:48.077343 16418 ts_tablet_manager.cc:1403] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.252s	user 0.012s	sys 0.026s
I20260709 13:06:48.079068 16288 tablet_service.cc:1511] Processing CreateTablet for tablet c66825803ddc46e8849c7718d3cb56ea (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:48.076823 16211 tablet_service.cc:1511] Processing CreateTablet for tablet 47e0028829c94c99a91ca4cc4d629a4a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:48.085137 16211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e0028829c94c99a91ca4cc4d629a4a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:48.085361 16325 tablet.cc:2406] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:06:48.089812 16280 tablet_service.cc:1511] Processing CreateTablet for tablet 2201023e16254eed9712e5fba24caa38 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:48.094866 16282 tablet_service.cc:1511] Processing CreateTablet for tablet 8b5ad93fb43d4467ac4dc1f70ccb0978 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:48.047411 16137 tablet_service.cc:1511] Processing CreateTablet for tablet 47e0028829c94c99a91ca4cc4d629a4a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:06:48.096942 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47e0028829c94c99a91ca4cc4d629a4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.097172 16289 tablet_service.cc:1511] Processing CreateTablet for tablet 76774e4153c7483683fcec5f557bfaad (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:48.100457 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c66825803ddc46e8849c7718d3cb56ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.102816 16286 tablet_service.cc:1511] Processing CreateTablet for tablet b521a909fd464b72bc2742ff3157658b (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:48.104730 16418 raft_consensus.cc:359] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:48.105536 16418 raft_consensus.cc:385] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.105846 16418 raft_consensus.cc:740] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.039568 16415 ts_tablet_manager.cc:1434] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.111s	user 0.008s	sys 0.006s
I20260709 13:06:48.121318 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b521a909fd464b72bc2742ff3157658b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.117161 16418 consensus_queue.cc:260] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:48.129175 16283 tablet_service.cc:1511] Processing CreateTablet for tablet 8a58bd7b2451404a8b70547dee94049a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:06:48.114104 16209 tablet_service.cc:1511] Processing CreateTablet for tablet c66825803ddc46e8849c7718d3cb56ea (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:06:48.132130 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c66825803ddc46e8849c7718d3cb56ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.141119 16206 tablet_service.cc:1511] Processing CreateTablet for tablet e652cb454b674e399c2ac22c22c05506 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:06:48.141644 16133 tablet_service.cc:1511] Processing CreateTablet for tablet 3c337868c6fc4e079f20a204e451ab66 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:48.142554 16206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e652cb454b674e399c2ac22c22c05506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.143337 16133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c337868c6fc4e079f20a204e451ab66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.146741 16416 tablet_bootstrap.cc:492] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:48.141044 16213 tablet_service.cc:1511] Processing CreateTablet for tablet ffb9eaed164d46ccac187f1878f24e4a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:06:48.152571 16212 tablet_service.cc:1511] Processing CreateTablet for tablet a042ac05958049c38414fa65d0c39414 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:06:48.160189 16207 tablet_service.cc:1511] Processing CreateTablet for tablet 76774e4153c7483683fcec5f557bfaad (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:48.163226 16416 tablet_bootstrap.cc:654] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.151701 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffb9eaed164d46ccac187f1878f24e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.168128 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a042ac05958049c38414fa65d0c39414. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:48.168576 16138 consensus_meta.cc:287] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.000s
I20260709 13:06:48.169932 16131 tablet_service.cc:1511] Processing CreateTablet for tablet 2201023e16254eed9712e5fba24caa38 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:48.171443 16131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2201023e16254eed9712e5fba24caa38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.182881 16138 tablet_service.cc:1511] Processing CreateTablet for tablet af66e3a8024e47fe925b7afad68d41c2 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:48.190914 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76774e4153c7483683fcec5f557bfaad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.213318 16132 tablet_service.cc:1511] Processing CreateTablet for tablet 76774e4153c7483683fcec5f557bfaad (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:06:48.203954 16424 raft_consensus.cc:493] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.216048 16424 raft_consensus.cc:515] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:48.223702 16138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af66e3a8024e47fe925b7afad68d41c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.228300 16140 tablet_service.cc:1511] Processing CreateTablet for tablet a6563b9b2e954646b0ce2535138676d0 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:06:48.236506 16424 leader_election.cc:290] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:48.237967 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6563b9b2e954646b0ce2535138676d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.150915 16135 tablet_service.cc:1511] Processing CreateTablet for tablet 8b5ad93fb43d4467ac4dc1f70ccb0978 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:06:48.244727 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5ad93fb43d4467ac4dc1f70ccb0978. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.261372 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76774e4153c7483683fcec5f557bfaad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.272871 16418 ts_tablet_manager.cc:1434] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.195s	user 0.014s	sys 0.011s
I20260709 13:06:48.274214 16418 tablet_bootstrap.cc:492] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:48.275096 16415 tablet_bootstrap.cc:492] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:48.276383 16285 tablet_service.cc:1511] Processing CreateTablet for tablet c0f16aeeb02645aea6c44a180eac4f87 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:48.277043 16289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76774e4153c7483683fcec5f557bfaad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.277911 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f16aeeb02645aea6c44a180eac4f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.282914 16214 tablet_service.cc:1511] Processing CreateTablet for tablet 3c337868c6fc4e079f20a204e451ab66 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:06:48.284963 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c337868c6fc4e079f20a204e451ab66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.285403 16288 tablet_service.cc:1511] Processing CreateTablet for tablet 18645784203b4ce1bbe3302513d910c1 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:48.276383 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 62345643f7684ec08f60f740de27cead (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:48.289019 16398 heartbeater.cc:499] Master 127.7.213.126:46449 was elected leader, sending a full tablet report...
I20260709 13:06:48.311691 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18645784203b4ce1bbe3302513d910c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.317714 16282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b5ad93fb43d4467ac4dc1f70ccb0978. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.320103 16283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a58bd7b2451404a8b70547dee94049a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.324039 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62345643f7684ec08f60f740de27cead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.327188 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b90ea5d39bbf4d79a4a44959a8f40359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.332844 16284 tablet_service.cc:1511] Processing CreateTablet for tablet 4cec4e308d104a9f9049fa0c2263b58d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:48.333498 16281 tablet_service.cc:1511] Processing CreateTablet for tablet e9db6747820949b6b76dcc3ab05485b1 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:48.334244 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cec4e308d104a9f9049fa0c2263b58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.334842 16281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9db6747820949b6b76dcc3ab05485b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.341774 16415 tablet_bootstrap.cc:654] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:48.346434 16139 consensus_meta.cc:287] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.004s
I20260709 13:06:48.358956 16280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2201023e16254eed9712e5fba24caa38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.362948 16355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b521a909fd464b72bc2742ff3157658b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:48.365350 16032 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 50621.5us
I20260709 13:06:48.369158 16363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abb840ca52d04206a82103fb4eb79ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.374066 16139 tablet_service.cc:1511] Processing CreateTablet for tablet c0f16aeeb02645aea6c44a180eac4f87 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:06:48.375442 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f16aeeb02645aea6c44a180eac4f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.381423 16417 ts_tablet_manager.cc:1434] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.431s	user 0.026s	sys 0.008s
I20260709 13:06:48.382587 16417 tablet_bootstrap.cc:492] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:48.391453 16359 tablet_service.cc:1511] Processing CreateTablet for tablet e1e24ed51cb74d0da9f8b77dcbdab5b9 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:48.396816 16359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e24ed51cb74d0da9f8b77dcbdab5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.398427 16428 raft_consensus.cc:493] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.400625 16428 raft_consensus.cc:515] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:48.403227 16428 leader_election.cc:290] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:48.399407 16416 tablet_bootstrap.cc:492] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:48.399849 16425 raft_consensus.cc:493] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.422391 16360 tablet_service.cc:1511] Processing CreateTablet for tablet 7186244ab72e4b32ad068941b913f052 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:48.424031 16416 ts_tablet_manager.cc:1403] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.277s	user 0.016s	sys 0.007s
I20260709 13:06:48.428630 16425 raft_consensus.cc:515] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:48.432850 16356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cec4e308d104a9f9049fa0c2263b58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.445986 16425 leader_election.cc:290] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:48.454600 16360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7186244ab72e4b32ad068941b913f052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.460572 16133 tablet_service.cc:1511] Processing CreateTablet for tablet abb840ca52d04206a82103fb4eb79ccf (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:48.463323 16357 tablet_service.cc:1511] Processing CreateTablet for tablet 42c540d71b564c08b9032559d7975cc5 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:48.439966 16136 tablet_service.cc:1511] Processing CreateTablet for tablet 62345643f7684ec08f60f740de27cead (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:48.516672 16133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abb840ca52d04206a82103fb4eb79ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.517736 16357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42c540d71b564c08b9032559d7975cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.552932 16418 tablet_bootstrap.cc:700] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Time spent opening tablet: real 0.149s	user 0.004s	sys 0.000s
I20260709 13:06:48.609229 16138 tablet_service.cc:1511] Processing CreateTablet for tablet 18645784203b4ce1bbe3302513d910c1 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:06:48.610363 16206 tablet_service.cc:1511] Processing CreateTablet for tablet b521a909fd464b72bc2742ff3157658b (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:06:48.612250 16206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b521a909fd464b72bc2742ff3157658b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.612097 16137 tablet_service.cc:1511] Processing CreateTablet for tablet e9db6747820949b6b76dcc3ab05485b1 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:48.612147 16213 tablet_service.cc:1511] Processing CreateTablet for tablet e9db6747820949b6b76dcc3ab05485b1 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:06:48.616907 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9db6747820949b6b76dcc3ab05485b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.623968 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62345643f7684ec08f60f740de27cead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.638255 16286 tablet_service.cc:1511] Processing CreateTablet for tablet e1e24ed51cb74d0da9f8b77dcbdab5b9 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:48.645650 16212 tablet_service.cc:1511] Processing CreateTablet for tablet abb840ca52d04206a82103fb4eb79ccf (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:06:48.647060 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abb840ca52d04206a82103fb4eb79ccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.648088 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9db6747820949b6b76dcc3ab05485b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.513278 16215 tablet_service.cc:1511] Processing CreateTablet for tablet 62345643f7684ec08f60f740de27cead (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:06:48.651973 16210 tablet_service.cc:1511] Processing CreateTablet for tablet 1dd94eae6c1241e5838930e00eb43c4c (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:48.652940 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62345643f7684ec08f60f740de27cead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.654434 16417 tablet_bootstrap.cc:654] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.661284 16207 tablet_service.cc:1511] Processing CreateTablet for tablet 7186244ab72e4b32ad068941b913f052 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:48.662679 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7186244ab72e4b32ad068941b913f052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.664065 16138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18645784203b4ce1bbe3302513d910c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.483693 16416 raft_consensus.cc:359] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:48.671975 16416 raft_consensus.cc:385] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:48.672353 16416 raft_consensus.cc:740] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.672756 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dd94eae6c1241e5838930e00eb43c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.674118 16134 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46886 (request call id 10) took 1033 ms. Trace:
I20260709 13:06:48.690725 16135 tablet_service.cc:1511] Processing CreateTablet for tablet e1e24ed51cb74d0da9f8b77dcbdab5b9 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:06:48.684682 16418 tablet_bootstrap.cc:654] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:48.487818 16214 tablet_service.cc:1511] Processing CreateTablet for tablet 7973b570c8554e048bd5a9d02fd13a46 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:48.674674 16134 rpcz_store.cc:276] 0709 13:06:47.640999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.716093 (+ 75094us) service_pool.cc:224] Handling call
0709 13:06:47.718076 (+  1983us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:47.718215 (+   139us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:48.435545 (+717330us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:48.673623 (+238078us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:48.674056 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.484540 16209 tablet_service.cc:1511] Processing CreateTablet for tablet af66e3a8024e47fe925b7afad68d41c2 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:06:48.531001 16208 tablet_service.cc:1511] Processing CreateTablet for tablet 4cec4e308d104a9f9049fa0c2263b58d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:06:48.516120 16211 tablet_service.cc:1511] Processing CreateTablet for tablet 2201023e16254eed9712e5fba24caa38 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:06:48.717064 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e24ed51cb74d0da9f8b77dcbdab5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.687093 16416 consensus_queue.cc:260] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:48.718485 16361 tablet_service.cc:1511] Processing CreateTablet for tablet 181b2ad7f8b649d4be93241715d03572 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:48.761590 16361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181b2ad7f8b649d4be93241715d03572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.739456 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:48.762003 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7973b570c8554e048bd5a9d02fd13a46. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:48.763329 16337 leader_election.cc:343] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.742044 16281 tablet_service.cc:1511] Processing CreateTablet for tablet 1dd94eae6c1241e5838930e00eb43c4c (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:48.765242 16139 tablet_service.cc:1511] Processing CreateTablet for tablet 181b2ad7f8b649d4be93241715d03572 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:48.766132 16281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dd94eae6c1241e5838930e00eb43c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.746815 16134 tablet_service.cc:1511] Processing CreateTablet for tablet 42c540d71b564c08b9032559d7975cc5 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:48.751492 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:48.764791 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 424dbaa8a798455b9ae4d2f0b0989561 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
W20260709 13:06:48.769379 16114 leader_election.cc:343] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.751960 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e24ed51cb74d0da9f8b77dcbdab5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.753201 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e218598a3114f679b2483143a2f4692" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:48.770375 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 424dbaa8a798455b9ae4d2f0b0989561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.774060 16140 tablet_service.cc:1511] Processing CreateTablet for tablet 7973b570c8554e048bd5a9d02fd13a46 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:06:48.775490 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7973b570c8554e048bd5a9d02fd13a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.812203 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e218598a3114f679b2483143a2f4692" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
W20260709 13:06:48.825075 16262 leader_election.cc:343] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.793473 16285 tablet_service.cc:1511] Processing CreateTablet for tablet 7186244ab72e4b32ad068941b913f052 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:06:48.785457 16360 tablet_service.cc:1511] Processing CreateTablet for tablet 0255ba6a147b416d8e3081a4e28bf626 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:48.826690 16360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0255ba6a147b416d8e3081a4e28bf626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.931250 16137 tablet_service.cc:1511] Processing CreateTablet for tablet 38bbd6b3bdff467f9d48329073cf0167 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:48.721643 16132 tablet_service.cc:1511] Processing CreateTablet for tablet 1dd94eae6c1241e5838930e00eb43c4c (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:06:48.776275 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af66e3a8024e47fe925b7afad68d41c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.799044 16288 tablet_service.cc:1511] Processing CreateTablet for tablet 58b7aa0c7f8b4bf4a2e4881b23351045 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:48.780285 16356 tablet_service.cc:1511] Processing CreateTablet for tablet 424dbaa8a798455b9ae4d2f0b0989561 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:48.761178 16284 tablet_service.cc:1511] Processing CreateTablet for tablet 181b2ad7f8b649d4be93241715d03572 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:06:48.874894 16136 tablet_service.cc:1511] Processing CreateTablet for tablet 0255ba6a147b416d8e3081a4e28bf626 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:48.831050 16417 tablet_bootstrap.cc:492] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
W20260709 13:06:48.923889 16032 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 52066.2us
I20260709 13:06:48.978758 16208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cec4e308d104a9f9049fa0c2263b58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.908969 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:48.855579 16282 tablet_service.cc:1511] Processing CreateTablet for tablet 38bbd6b3bdff467f9d48329073cf0167 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:06:48.860976 16207 tablet_service.cc:1511] Processing CreateTablet for tablet 42c540d71b564c08b9032559d7975cc5 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:06:48.776715 16225 raft_consensus.cc:2468] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:48.860219 16206 tablet_service.cc:1511] Processing CreateTablet for tablet 58b7aa0c7f8b4bf4a2e4881b23351045 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:48.900820 16131 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46886 (request call id 21) took 1024 ms. Trace:
I20260709 13:06:48.831863 16138 tablet_service.cc:1511] Processing CreateTablet for tablet 58b7aa0c7f8b4bf4a2e4881b23351045 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:06:48.922881 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7186244ab72e4b32ad068941b913f052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.995147 16262 leader_election.cc:304] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: c8106927ed954e1c800869427b6c7964
W20260709 13:06:48.854592 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:48.837493 16442 raft_consensus.cc:493] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:48.828264 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:48.995141 16131 rpcz_store.cc:276] 0709 13:06:47.876718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.168956 (+292238us) service_pool.cc:224] Handling call
0709 13:06:48.170823 (+  1867us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:48.170922 (+    99us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:48.746803 (+575881us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:48.891692 (+144889us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:48.900733 (+  9041us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:48.842434 16415 tablet_bootstrap.cc:492] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:48.996348 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 1047f65a57d24cafbf3f41f5b235556e (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:48.997608 16415 ts_tablet_manager.cc:1403] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.723s	user 0.040s	sys 0.061s
I20260709 13:06:48.998531 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b7aa0c7f8b4bf4a2e4881b23351045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.998907 16361 tablet_service.cc:1511] Processing CreateTablet for tablet 38bbd6b3bdff467f9d48329073cf0167 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
W20260709 13:06:48.924193 16033 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 46644.3us
I20260709 13:06:48.918924 16418 tablet_bootstrap.cc:492] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:49.001327 16210 tablet_service.cc:1511] Processing CreateTablet for tablet 26c52cafca664c5f90589901f62e4b16 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:49.002014 16418 ts_tablet_manager.cc:1403] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.728s	user 0.011s	sys 0.019s
I20260709 13:06:49.003423 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181b2ad7f8b649d4be93241715d03572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.004221 16282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38bbd6b3bdff467f9d48329073cf0167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.829619 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42c540d71b564c08b9032559d7975cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.004335 16140 tablet_service.cc:1511] Processing CreateTablet for tablet 1047f65a57d24cafbf3f41f5b235556e (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:06:49.005687 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1047f65a57d24cafbf3f41f5b235556e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.868868 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 424dbaa8a798455b9ae4d2f0b0989561 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:06:49.006604 16214 tablet_service.cc:1511] Processing CreateTablet for tablet 1dea723251ed4848b42709f873b7d446 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:48.957911 16215 tablet_service.cc:1511] Processing CreateTablet for tablet 0255ba6a147b416d8e3081a4e28bf626 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:06:48.973644 16133 tablet_service.cc:1511] Processing CreateTablet for tablet 26c52cafca664c5f90589901f62e4b16 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:49.008248 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0255ba6a147b416d8e3081a4e28bf626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.979640 16356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 424dbaa8a798455b9ae4d2f0b0989561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.986972 16417 ts_tablet_manager.cc:1403] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.605s	user 0.020s	sys 0.009s
I20260709 13:06:49.012010 16361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38bbd6b3bdff467f9d48329073cf0167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.012115 16138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b7aa0c7f8b4bf4a2e4881b23351045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.985920 16355 tablet_service.cc:1511] Processing CreateTablet for tablet 1047f65a57d24cafbf3f41f5b235556e (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:06:49.012625 16338 leader_election.cc:343] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:48.989045 16357 tablet_service.cc:1511] Processing CreateTablet for tablet 1dea723251ed4848b42709f873b7d446 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:49.013154 16338 leader_election.cc:304] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:49.013914 16355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1047f65a57d24cafbf3f41f5b235556e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.014925 16133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c52cafca664c5f90589901f62e4b16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.016091 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38bbd6b3bdff467f9d48329073cf0167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.017877 16357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dea723251ed4848b42709f873b7d446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:48.996189 16442 raft_consensus.cc:515] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.019706 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dd94eae6c1241e5838930e00eb43c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.023231 16286 tablet_service.cc:1511] Processing CreateTablet for tablet 1dea723251ed4848b42709f873b7d446 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:06:49.023980 16224 raft_consensus.cc:2468] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:49.024739 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dea723251ed4848b42709f873b7d446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.032198 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181b2ad7f8b649d4be93241715d03572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.041541 16415 raft_consensus.cc:359] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.042340 16415 raft_consensus.cc:385] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.042237 16281 tablet_service.cc:1511] Processing CreateTablet for tablet 72031807a36e49fa81fe0eb9fb94a7a0 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:49.042745 16415 raft_consensus.cc:740] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.043717 16281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72031807a36e49fa81fe0eb9fb94a7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.044835 16418 raft_consensus.cc:359] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.045598 16418 raft_consensus.cc:385] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.045892 16418 raft_consensus.cc:740] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.046702 16418 consensus_queue.cc:260] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.048425 16442 leader_election.cc:290] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:49.051716 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 26c52cafca664c5f90589901f62e4b16 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:06:49.056516 16360 tablet_service.cc:1511] Processing CreateTablet for tablet 72031807a36e49fa81fe0eb9fb94a7a0 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:49.057725 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c52cafca664c5f90589901f62e4b16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.057868 16360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72031807a36e49fa81fe0eb9fb94a7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.064862 16417 raft_consensus.cc:359] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:49.070413 16131 tablet_service.cc:1511] Processing CreateTablet for tablet 72031807a36e49fa81fe0eb9fb94a7a0 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:06:49.075187 16417 raft_consensus.cc:385] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.075938 16131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72031807a36e49fa81fe0eb9fb94a7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.043531 16415 consensus_queue.cc:260] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.075590 16417 raft_consensus.cc:740] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:48.983914 16211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2201023e16254eed9712e5fba24caa38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.085457 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0255ba6a147b416d8e3081a4e28bf626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.091817 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42c540d71b564c08b9032559d7975cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.092578 16417 consensus_queue.cc:260] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:49.093603 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dea723251ed4848b42709f873b7d446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.097802 16445 raft_consensus.cc:2804] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.127836 16445 raft_consensus.cc:493] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.128376 16445 raft_consensus.cc:3060] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:49.129101 16320 debug-util.cc:398] Leaking SignalData structure 0x7b080018e560 after lost signal to thread 8024
W20260709 13:06:49.130426 16320 debug-util.cc:398] Leaking SignalData structure 0x7b080016eec0 after lost signal to thread 16082
I20260709 13:06:49.130822 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 424dbaa8a798455b9ae4d2f0b0989561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.115343 16415 ts_tablet_manager.cc:1434] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.117s	user 0.007s	sys 0.003s
I20260709 13:06:49.132565 16415 tablet_bootstrap.cc:492] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:49.135936 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1047f65a57d24cafbf3f41f5b235556e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.120136 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 125085us
I20260709 13:06:49.138806 16113 leader_election.cc:304] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:49.146847 16415 tablet_bootstrap.cc:654] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.146848 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26c52cafca664c5f90589901f62e4b16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.153563 16289 tablet_service.cc:1511] Processing CreateTablet for tablet 30b15e00d79d42809ddd6a31dd9ee9f3 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:49.154886 16289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30b15e00d79d42809ddd6a31dd9ee9f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.160099 16206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58b7aa0c7f8b4bf4a2e4881b23351045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.168568 16452 raft_consensus.cc:2749] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.195518 16417 ts_tablet_manager.cc:1434] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.184s	user 0.006s	sys 0.005s
I20260709 13:06:49.226177 16453 raft_consensus.cc:2804] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.230576 16453 raft_consensus.cc:493] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.231011 16453 raft_consensus.cc:3060] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:49.208505 16113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 69841.3us
I20260709 13:06:49.246054 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:49.254932 16189 leader_election.cc:343] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.255677 16356 tablet_service.cc:1511] Processing CreateTablet for tablet 30b15e00d79d42809ddd6a31dd9ee9f3 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:49.256341 16131 tablet_service.cc:1511] Processing CreateTablet for tablet a6959f2cf9714e91aee31bf53776915b (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260709 13:06:49.250681 16338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 238092us
I20260709 13:06:49.263499 16418 ts_tablet_manager.cc:1434] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.261s	user 0.009s	sys 0.018s
I20260709 13:06:49.252409 16362 tablet_service.cc:1511] Processing CreateTablet for tablet a6959f2cf9714e91aee31bf53776915b (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:49.272148 16284 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba3cb60361b431ba89f7842cdb1c85a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:49.275979 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:49.278398 16131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6959f2cf9714e91aee31bf53776915b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.281164 16415 tablet_bootstrap.cc:492] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:49.281690 16415 ts_tablet_manager.cc:1403] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.149s	user 0.010s	sys 0.008s
I20260709 13:06:49.282584 16418 tablet_bootstrap.cc:492] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:49.287361 16208 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36054 (request call id 26) took 1155 ms. Trace:
I20260709 13:06:49.289685 16416 ts_tablet_manager.cc:1434] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.865s	user 0.019s	sys 0.032s
I20260709 13:06:49.287568 16417 tablet_bootstrap.cc:492] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:49.257151 16356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30b15e00d79d42809ddd6a31dd9ee9f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.306367 16416 ts_tablet_manager.cc:1467] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776: Tablet startup took 881ms
W20260709 13:06:49.307678 16416 ts_tablet_manager.cc:1469] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776: Trace:
0709 13:06:47.456001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:47.463854 (+  7853us) service_pool.cc:224] Handling call
0709 13:06:47.466039 (+  2185us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:47.466160 (+   121us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:47.490188 (+ 24028us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:47.624748 (+134560us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:47.625218 (+   470us) inbound_call.cc:177] Queueing success response
0709 13:06:48.146391 (+521173us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:06:48.424609 (+278218us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:06:48.425403 (+   794us) tablet_replica.cc:261] Starting instrumentation
0709 13:06:48.426393 (+   990us) tablet_replica.cc:280] Starting consensus
0709 13:06:48.997612 (+571219us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400695854. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765664d571 00007f76566db12e 00007f76566d9c8d 00007f765a0d6b2e 00007f765b6f8d8a 00007f765b70cd01 00007f765b70cca9 00007f765b70cc39 00007f765b70cc01 00007f765b70bedd 000000000038f354
Metrics: {"raft.queue_time_us":51255,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2136,"spinlock_wait_cycles":486831360,"tablet-open.queue_time_us":521260,"thread_start_us":224001,"threads_started":1}
I20260709 13:06:49.312510 16362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6959f2cf9714e91aee31bf53776915b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.323145 16215 tablet_service.cc:1511] Processing CreateTablet for tablet 30b15e00d79d42809ddd6a31dd9ee9f3 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:06:49.325809 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30b15e00d79d42809ddd6a31dd9ee9f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.319557 16415 raft_consensus.cc:359] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.326402 16415 raft_consensus.cc:385] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.326725 16415 raft_consensus.cc:740] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.327481 16415 consensus_queue.cc:260] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.329630 16415 ts_tablet_manager.cc:1434] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:06:49.322140 16417 tablet_bootstrap.cc:654] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.321877 16453 raft_consensus.cc:515] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.323320 16285 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:37794 (request call id 29) took 1068 ms. Trace:
I20260709 13:06:49.330258 16285 rpcz_store.cc:276] 0709 13:06:48.260013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.726933 (+466920us) service_pool.cc:224] Handling call
0709 13:06:48.922248 (+195315us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:48.922377 (+   129us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:49.078425 (+156048us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:49.322803 (+244378us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:49.323262 (+   459us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26692,"spinlock_wait_cycles":30976}
I20260709 13:06:49.332953 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba3cb60361b431ba89f7842cdb1c85a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.331768 16150 raft_consensus.cc:2393] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce6da6d71124256be0679451014f776 in current term 1: Already voted for candidate aa8eb07867e7420b80a7e615d570509d in this term.
I20260709 13:06:49.336377 16453 leader_election.cc:290] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:49.336191 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:49.337893 16224 raft_consensus.cc:3060] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.338260 16190 leader_election.cc:304] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:49.338569 16416 tablet_bootstrap.cc:492] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:49.340304 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:49.306896 16445 raft_consensus.cc:515] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
W20260709 13:06:49.345016 16114 leader_election.cc:343] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.357084 16418 tablet_bootstrap.cc:654] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.359519 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e218598a3114f679b2483143a2f4692" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:49.360172 16225 raft_consensus.cc:3060] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.363571 16445 leader_election.cc:290] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:49.351513 16224 raft_consensus.cc:2468] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:49.353286 16285 tablet_service.cc:1511] Processing CreateTablet for tablet de7d234ecf1f47fabae92b3be1d2b84d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:49.364431 16446 raft_consensus.cc:2749] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.364790 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e218598a3114f679b2483143a2f4692" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:49.365450 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7d234ecf1f47fabae92b3be1d2b84d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:06:49.366338 16262 leader_election.cc:343] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.367151 16135 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:46886 (request call id 29) took 1151 ms. Trace:
I20260709 13:06:49.367419 16113 leader_election.cc:304] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:49.372819 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba3cb60361b431ba89f7842cdb1c85a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:49.374559 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba3cb60361b431ba89f7842cdb1c85a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.376209 16453 raft_consensus.cc:2804] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.346134 16355 tablet_service.cc:1511] Processing CreateTablet for tablet de7d234ecf1f47fabae92b3be1d2b84d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:49.305708 16208 rpcz_store.cc:276] 0709 13:06:48.131773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.529694 (+397921us) service_pool.cc:224] Handling call
0709 13:06:48.786374 (+256680us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:48.786521 (+   147us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:49.151214 (+364693us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:49.286886 (+135672us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:49.287301 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:49.379822 16453 raft_consensus.cc:697] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:49.367592 16135 rpcz_store.cc:276] 0709 13:06:48.215723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.689748 (+474025us) service_pool.cc:224] Handling call
0709 13:06:48.691629 (+  1881us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:48.691733 (+   104us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:49.125603 (+433870us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:49.366658 (+241055us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:49.367086 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:49.380821 16355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7d234ecf1f47fabae92b3be1d2b84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.372419 16206 tablet_service.cc:1511] Processing CreateTablet for tablet a6959f2cf9714e91aee31bf53776915b (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:06:49.381783 16453 consensus_queue.cc:237] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.382725 16206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6959f2cf9714e91aee31bf53776915b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.385620 16225 raft_consensus.cc:2468] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:49.386658 16211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36054 (request call id 21) took 1270 ms. Trace:
I20260709 13:06:49.386955 16262 leader_election.cc:304] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: c8106927ed954e1c800869427b6c7964
I20260709 13:06:49.387080 16211 rpcz_store.cc:276] 0709 13:06:48.130028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.384229 (+254201us) service_pool.cc:224] Handling call
0709 13:06:48.786168 (+401939us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b5800322298. stack: 00007f76535c5343 00007f76535c5240 00007f7652f541d4 00007f765cc92304 00007f765cc92299 00007f765347fdb1 00007f76535bfe9c 00007f76535bfa5b 00007f765c4d8c09 00007f765b632ee0 00007f765b718831 00007f765b6f5ac3 00007f765b6a4400 00007f7654b29af4 00007f7654b29a82 00007f7654b299b1
0709 13:06:48.823157 (+ 36989us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:48.823269 (+   112us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:49.178359 (+355090us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:49.381253 (+202894us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:49.386603 (+  5350us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4892,"spinlock_wait_cycles":13741952}
I20260709 13:06:49.340794 16134 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba3cb60361b431ba89f7842cdb1c85a (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:06:49.390568 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba3cb60361b431ba89f7842cdb1c85a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.391299 16209 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36054 (request call id 22) took 1260 ms. Trace:
I20260709 13:06:49.391729 16209 rpcz_store.cc:276] 0709 13:06:48.130466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:06:48.433670 (+303204us) service_pool.cc:224] Handling call
0709 13:06:48.758948 (+325278us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:06:48.775798 (+ 16850us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:06:49.274277 (+498479us) tablet_metadata.cc:665] Metadata flushed
0709 13:06:49.390819 (+116542us) tablet_replica.cc:206] Creating consensus instance
0709 13:06:49.391240 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:06:49.389497 16445 raft_consensus.cc:2804] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.393849 16445 raft_consensus.cc:697] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:49.377494 16416 tablet_bootstrap.cc:654] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.394757 16445 consensus_queue.cc:237] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:49.397395 16415 tablet_bootstrap.cc:492] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:49.409937 16417 tablet_bootstrap.cc:492] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:49.410523 16417 ts_tablet_manager.cc:1403] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.123s	user 0.018s	sys 0.000s
I20260709 13:06:49.413625 16417 raft_consensus.cc:359] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.414352 16417 raft_consensus.cc:385] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.422286 16140 tablet_service.cc:1511] Processing CreateTablet for tablet de7d234ecf1f47fabae92b3be1d2b84d (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:06:49.423257 16287 tablet_service.cc:1511] Processing CreateTablet for tablet c0cbd7d8de354586aab3116154d3ee3c (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:49.423715 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7d234ecf1f47fabae92b3be1d2b84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.424532 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0cbd7d8de354586aab3116154d3ee3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.431030 16209 tablet_service.cc:1511] Processing CreateTablet for tablet c0cbd7d8de354586aab3116154d3ee3c (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:49.432525 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0cbd7d8de354586aab3116154d3ee3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.434890 16361 tablet_service.cc:1511] Processing CreateTablet for tablet c0cbd7d8de354586aab3116154d3ee3c (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:06:49.414675 16417 raft_consensus.cc:740] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.444602 16361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0cbd7d8de354586aab3116154d3ee3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.445883 16137 tablet_service.cc:1511] Processing CreateTablet for tablet 6714e2c053f14fbeaec12683da401eb4 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:49.452884 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6714e2c053f14fbeaec12683da401eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.448702 16415 tablet_bootstrap.cc:654] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.450758 16416 tablet_bootstrap.cc:492] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:49.460641 16447 raft_consensus.cc:493] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.461164 16447 raft_consensus.cc:515] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:49.462713 16360 tablet_service.cc:1511] Processing CreateTablet for tablet 09715ac5d7d741e3aabf462dcab633ba (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:49.464083 16360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09715ac5d7d741e3aabf462dcab633ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.464766 16135 tablet_service.cc:1511] Processing CreateTablet for tablet 09715ac5d7d741e3aabf462dcab633ba (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:49.465987 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09715ac5d7d741e3aabf462dcab633ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.468309 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:49.468865 16417 consensus_queue.cc:260] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.462713 16286 tablet_service.cc:1511] Processing CreateTablet for tablet 6714e2c053f14fbeaec12683da401eb4 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:49.470659 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6714e2c053f14fbeaec12683da401eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.470089 16044 catalog_manager.cc:5697] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:49.472288 16338 leader_election.cc:343] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.463464 16447 leader_election.cc:290] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:49.480940 16211 tablet_service.cc:1511] Processing CreateTablet for tablet 6714e2c053f14fbeaec12683da401eb4 (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:06:49.482611 16211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6714e2c053f14fbeaec12683da401eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.495558 16131 tablet_service.cc:1511] Processing CreateTablet for tablet 819e82eed43e4968984673887beb39ec (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:49.477487 16453 raft_consensus.cc:493] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.497185 16131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819e82eed43e4968984673887beb39ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.497470 16453 raft_consensus.cc:515] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.498893 16046 catalog_manager.cc:5697] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.495527 16284 tablet_service.cc:1511] Processing CreateTablet for tablet 819e82eed43e4968984673887beb39ec (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:49.473129 16416 ts_tablet_manager.cc:1403] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.135s	user 0.012s	sys 0.009s
I20260709 13:06:49.505956 16453 leader_election.cc:290] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:49.511135 16210 tablet_service.cc:1511] Processing CreateTablet for tablet 09715ac5d7d741e3aabf462dcab633ba (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:06:49.517736 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b5e6c6090448ab831f8afeef43db0d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:49.522383 16298 raft_consensus.cc:2468] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:49.522924 16417 ts_tablet_manager.cc:1434] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.112s	user 0.006s	sys 0.004s
I20260709 13:06:49.523880 16114 leader_election.cc:304] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:49.524009 16355 tablet_service.cc:1511] Processing CreateTablet for tablet 819e82eed43e4968984673887beb39ec (DEFAULT_TABLE table=test-workload [id=3541558a29db4bdfb0c2f14b3e3fad67]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:06:49.525437 16355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819e82eed43e4968984673887beb39ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.525864 16453 raft_consensus.cc:2804] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.526298 16453 raft_consensus.cc:493] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.526619 16453 raft_consensus.cc:3060] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.536276 16417 tablet_bootstrap.cc:492] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:49.545929 16416 raft_consensus.cc:359] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.546573 16416 raft_consensus.cc:385] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.546917 16416 raft_consensus.cc:740] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.557590 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 819e82eed43e4968984673887beb39ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.564368 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09715ac5d7d741e3aabf462dcab633ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:06:49.565709 16418 tablet_bootstrap.cc:492] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:49.566221 16418 ts_tablet_manager.cc:1403] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.284s	user 0.013s	sys 0.020s
I20260709 13:06:49.566171 16453 raft_consensus.cc:515] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.574862 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b5e6c6090448ab831f8afeef43db0d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:49.575428 16453 leader_election.cc:290] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:49.575531 16298 raft_consensus.cc:3060] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.584192 16298 raft_consensus.cc:2468] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:49.585517 16114 leader_election.cc:304] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:49.586404 16450 raft_consensus.cc:2804] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.547710 16416 consensus_queue.cc:260] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.589881 16416 ts_tablet_manager.cc:1434] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.085s	user 0.005s	sys 0.000s
I20260709 13:06:49.592255 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b5e6c6090448ab831f8afeef43db0d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:49.592950 16373 raft_consensus.cc:2468] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:49.594054 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b5e6c6090448ab831f8afeef43db0d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:49.594686 16372 raft_consensus.cc:3060] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.596076 16445 raft_consensus.cc:493] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.595857 16418 raft_consensus.cc:359] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.596735 16418 raft_consensus.cc:385] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.596557 16445 raft_consensus.cc:515] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.597045 16418 raft_consensus.cc:740] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.605021 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:49.605697 16373 raft_consensus.cc:2468] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:49.600142 16416 tablet_bootstrap.cc:492] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:49.611941 16262 leader_election.cc:304] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:49.612952 16451 raft_consensus.cc:2804] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.613332 16451 raft_consensus.cc:493] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.613785 16415 tablet_bootstrap.cc:492] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:49.614311 16415 ts_tablet_manager.cc:1403] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.217s	user 0.013s	sys 0.008s
I20260709 13:06:49.602149 16445 leader_election.cc:290] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:49.613638 16451 raft_consensus.cc:3060] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.624155 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:06:49.628501 16336 leader_election.cc:343] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.629019 16336 leader_election.cc:304] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:49.630178 16452 raft_consensus.cc:2749] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.634055 16451 raft_consensus.cc:515] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.635118 16372 raft_consensus.cc:2468] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:49.640947 16417 tablet_bootstrap.cc:654] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.597960 16418 consensus_queue.cc:260] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.643894 16416 tablet_bootstrap.cc:654] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.646353 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:49.647274 16451 leader_election.cc:290] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:49.628638 16415 raft_consensus.cc:359] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.649514 16453 raft_consensus.cc:493] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.651043 16372 raft_consensus.cc:3060] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.636797 16450 raft_consensus.cc:697] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:49.660439 16450 consensus_queue.cc:237] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.653257 16418 ts_tablet_manager.cc:1434] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.087s	user 0.009s	sys 0.000s
I20260709 13:06:49.663818 16453 raft_consensus.cc:515] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.678411 16418 tablet_bootstrap.cc:492] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:49.683511 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228d4a700de40ada51433b8043bf6a7" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:06:49.685330 16113 leader_election.cc:343] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.659687 16415 raft_consensus.cc:385] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.686736 16415 raft_consensus.cc:740] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.687727 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228d4a700de40ada51433b8043bf6a7" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:49.687808 16415 consensus_queue.cc:260] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:06:49.689386 16114 leader_election.cc:343] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.689842 16114 leader_election.cc:304] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:49.690456 16415 ts_tablet_manager.cc:1434] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.076s	user 0.009s	sys 0.000s
I20260709 13:06:49.690874 16466 raft_consensus.cc:2749] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:49.691562 16415 tablet_bootstrap.cc:492] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:49.693742 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:49.695348 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:49.691946 16453 leader_election.cc:290] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:49.710206 16416 tablet_bootstrap.cc:492] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:49.710851 16416 ts_tablet_manager.cc:1403] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.120s	user 0.033s	sys 0.011s
I20260709 13:06:49.712414 16418 tablet_bootstrap.cc:654] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.715149 16372 raft_consensus.cc:2468] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:49.719650 16262 leader_election.cc:304] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:49.720604 16451 raft_consensus.cc:2804] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.721100 16451 raft_consensus.cc:697] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:49.721915 16451 consensus_queue.cc:237] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.722852 16415 tablet_bootstrap.cc:654] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:49.723894 16264 leader_election.cc:343] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:49.725023 16264 leader_election.cc:343] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.725276 16417 tablet_bootstrap.cc:492] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:49.726400 16417 ts_tablet_manager.cc:1403] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.190s	user 0.008s	sys 0.015s
I20260709 13:06:49.731405 16416 raft_consensus.cc:359] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.732518 16416 raft_consensus.cc:385] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.732899 16416 raft_consensus.cc:740] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.733881 16416 consensus_queue.cc:260] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.738862 16417 raft_consensus.cc:359] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:49.739902 16417 raft_consensus.cc:385] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.740289 16417 raft_consensus.cc:740] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.741474 16417 consensus_queue.cc:260] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:49.748741 16416 ts_tablet_manager.cc:1434] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.038s	user 0.011s	sys 0.000s
I20260709 13:06:49.750303 16416 tablet_bootstrap.cc:492] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:49.757099 16417 ts_tablet_manager.cc:1434] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.004s
I20260709 13:06:49.758610 16417 tablet_bootstrap.cc:492] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:49.765367 16418 tablet_bootstrap.cc:492] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:49.765977 16418 ts_tablet_manager.cc:1403] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.088s	user 0.012s	sys 0.007s
I20260709 13:06:49.767414 16416 tablet_bootstrap.cc:654] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.770918 16451 raft_consensus.cc:493] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.771811 16451 raft_consensus.cc:515] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.777984 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:49.778705 16225 raft_consensus.cc:2468] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:49.779335 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:49.781185 16264 leader_election.cc:343] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.782459 16262 leader_election.cc:304] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:49.783437 16445 raft_consensus.cc:2804] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:49.783908 16445 raft_consensus.cc:493] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:49.784364 16445 raft_consensus.cc:3060] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.791849 16451 leader_election.cc:290] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:49.775061 16415 tablet_bootstrap.cc:492] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:49.792665 16417 tablet_bootstrap.cc:654] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.793237 16415 ts_tablet_manager.cc:1403] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.102s	user 0.009s	sys 0.009s
I20260709 13:06:49.797952 16415 raft_consensus.cc:359] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.802155 16415 raft_consensus.cc:385] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.802596 16415 raft_consensus.cc:740] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.803571 16415 consensus_queue.cc:260] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.800350 16445 raft_consensus.cc:515] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.801342 16418 raft_consensus.cc:359] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.812038 16418 raft_consensus.cc:385] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.812484 16418 raft_consensus.cc:740] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.814503 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
W20260709 13:06:49.818181 16264 leader_election.cc:343] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.821383 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:49.822305 16225 raft_consensus.cc:3060] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.823297 16418 consensus_queue.cc:260] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.824908 16451 consensus_queue.cc:1048] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:49.825707 16418 ts_tablet_manager.cc:1434] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 13:06:49.826625 16445 leader_election.cc:290] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:49.836097 16418 tablet_bootstrap.cc:492] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:49.836642 16415 ts_tablet_manager.cc:1434] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:06:49.838130 16415 tablet_bootstrap.cc:492] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:49.843811 16225 raft_consensus.cc:2468] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:49.848212 16417 tablet_bootstrap.cc:492] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:49.847999 16262 leader_election.cc:304] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:49.848887 16417 ts_tablet_manager.cc:1403] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.091s	user 0.037s	sys 0.008s
I20260709 13:06:49.844748 16415 tablet_bootstrap.cc:654] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.849537 16445 raft_consensus.cc:2804] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:49.850102 16445 raft_consensus.cc:697] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:49.850244 16046 catalog_manager.cc:5697] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.851128 16445 consensus_queue.cc:237] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.852219 16417 raft_consensus.cc:359] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.852985 16417 raft_consensus.cc:385] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.853277 16417 raft_consensus.cc:740] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.856231 16044 catalog_manager.cc:5697] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:49.857281 16416 tablet_bootstrap.cc:492] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:49.858085 16416 ts_tablet_manager.cc:1403] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.108s	user 0.023s	sys 0.011s
I20260709 13:06:49.867857 16417 consensus_queue.cc:260] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.871127 16417 ts_tablet_manager.cc:1434] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.022s	user 0.002s	sys 0.005s
I20260709 13:06:49.877406 16416 raft_consensus.cc:359] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:49.878681 16416 raft_consensus.cc:385] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.879119 16416 raft_consensus.cc:740] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
W20260709 13:06:49.879681 16114 consensus_peers.cc:597] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:49.889060 16417 tablet_bootstrap.cc:492] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:49.896818 16416 consensus_queue.cc:260] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:49.875295 16418 tablet_bootstrap.cc:654] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.912490 16416 ts_tablet_manager.cc:1434] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.054s	user 0.012s	sys 0.000s
I20260709 13:06:49.913801 16416 tablet_bootstrap.cc:492] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:49.916112 16418 tablet_bootstrap.cc:492] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:49.918270 16415 tablet_bootstrap.cc:492] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:49.918870 16415 ts_tablet_manager.cc:1403] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.081s	user 0.040s	sys 0.024s
I20260709 13:06:49.921197 16418 ts_tablet_manager.cc:1403] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.093s	user 0.013s	sys 0.003s
I20260709 13:06:49.926745 16415 raft_consensus.cc:359] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:49.927699 16415 raft_consensus.cc:385] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.928092 16415 raft_consensus.cc:740] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.930087 16418 raft_consensus.cc:359] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.929804 16045 catalog_manager.cc:5697] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:49.931552 16417 tablet_bootstrap.cc:654] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.934204 16418 raft_consensus.cc:385] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.933023 16415 consensus_queue.cc:260] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:49.938277 16415 ts_tablet_manager.cc:1434] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.001s
I20260709 13:06:49.940181 16416 tablet_bootstrap.cc:654] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.942555 16415 tablet_bootstrap.cc:492] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:49.935580 16418 raft_consensus.cc:740] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:49.946926 16418 consensus_queue.cc:260] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.950645 16418 ts_tablet_manager.cc:1434] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 13:06:49.954329 16418 tablet_bootstrap.cc:492] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:49.965301 16415 tablet_bootstrap.cc:654] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.972644 16418 tablet_bootstrap.cc:654] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:49.973047 16416 tablet_bootstrap.cc:492] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:49.973312 16446 raft_consensus.cc:493] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:49.973719 16416 ts_tablet_manager.cc:1403] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.001s
I20260709 13:06:49.973927 16446 raft_consensus.cc:515] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:49.978806 16416 raft_consensus.cc:359] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.979216 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd508336a7c844cc84117b7ef3a565ba" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:49.979719 16416 raft_consensus.cc:385] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:49.982287 16446 leader_election.cc:290] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:49.984977 16416 raft_consensus.cc:740] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
W20260709 13:06:49.982169 16190 leader_election.cc:343] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:49.984325 16417 tablet_bootstrap.cc:492] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:49.987526 16417 ts_tablet_manager.cc:1403] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.099s	user 0.017s	sys 0.002s
I20260709 13:06:49.991163 16415 tablet_bootstrap.cc:492] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:49.991837 16418 tablet_bootstrap.cc:492] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:49.992331 16418 ts_tablet_manager.cc:1403] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.006s
I20260709 13:06:49.992449 16415 ts_tablet_manager.cc:1403] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.005s
I20260709 13:06:49.994144 16372 raft_consensus.cc:3060] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:49.986002 16416 consensus_queue.cc:260] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:49.999135 16416 ts_tablet_manager.cc:1434] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.006s
I20260709 13:06:49.991377 16417 raft_consensus.cc:359] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.000198 16417 raft_consensus.cc:385] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.000504 16417 raft_consensus.cc:740] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.001405 16417 consensus_queue.cc:260] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
W20260709 13:06:50.002751 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:50.003012 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd508336a7c844cc84117b7ef3a565ba" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.006075 16415 raft_consensus.cc:359] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.006959 16415 raft_consensus.cc:385] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:06:50.006882 16188 leader_election.cc:343] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.007475 16415 raft_consensus.cc:740] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.007651 16188 leader_election.cc:304] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.008543 16446 raft_consensus.cc:2749] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.009414 16415 consensus_queue.cc:260] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.009703 16418 raft_consensus.cc:359] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.010668 16418 raft_consensus.cc:385] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.010191 16483 consensus_queue.cc:1048] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.011142 16418 raft_consensus.cc:740] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.012112 16415 ts_tablet_manager.cc:1434] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.003s
I20260709 13:06:50.012387 16450 consensus_queue.cc:1048] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.012172 16418 consensus_queue.cc:260] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.013520 16415 tablet_bootstrap.cc:492] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:50.015086 16418 ts_tablet_manager.cc:1434] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:06:50.016436 16418 tablet_bootstrap.cc:492] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:50.008244 16417 ts_tablet_manager.cc:1434] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:06:50.017697 16417 tablet_bootstrap.cc:492] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:50.020522 16416 tablet_bootstrap.cc:492] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:50.021699 16415 tablet_bootstrap.cc:654] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.024797 16418 tablet_bootstrap.cc:654] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.041882 16418 tablet_bootstrap.cc:492] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:50.042369 16418 ts_tablet_manager.cc:1403] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.004s
I20260709 13:06:50.092397 16416 tablet_bootstrap.cc:654] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.096401 16446 raft_consensus.cc:493] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.101336 16446 raft_consensus.cc:515] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.107609 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.109289 16373 raft_consensus.cc:2468] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:50.111130 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:50.111711 16188 leader_election.cc:304] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:50.111428 16415 tablet_bootstrap.cc:492] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
W20260709 13:06:50.116772 16190 leader_election.cc:343] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.127413 16415 ts_tablet_manager.cc:1403] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.114s	user 0.021s	sys 0.009s
I20260709 13:06:50.128482 16452 raft_consensus.cc:493] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.129546 16452 raft_consensus.cc:515] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.134385 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9a1ca7eb1348438944c99fcc3eb8e9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:50.135466 16299 raft_consensus.cc:2468] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:50.137557 16337 leader_election.cc:304] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:50.139345 16450 consensus_queue.cc:1048] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.148003 16453 consensus_queue.cc:1048] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:50.156255 16188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 44782.4us
I20260709 13:06:50.156936 16498 raft_consensus.cc:2804] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.157579 16498 raft_consensus.cc:493] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.158054 16498 raft_consensus.cc:3060] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.119853 16446 leader_election.cc:290] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.133585 16415 raft_consensus.cc:359] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.161705 16415 raft_consensus.cc:385] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.162148 16415 raft_consensus.cc:740] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.163105 16415 consensus_queue.cc:260] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.167327 16452 leader_election.cc:290] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.167970 16452 raft_consensus.cc:2804] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.168418 16452 raft_consensus.cc:493] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.168833 16452 raft_consensus.cc:3060] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.170456 16418 raft_consensus.cc:359] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.172824 16418 raft_consensus.cc:385] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.173167 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9a1ca7eb1348438944c99fcc3eb8e9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:50.173240 16418 raft_consensus.cc:740] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.174364 16149 raft_consensus.cc:2468] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:50.176843 16417 tablet_bootstrap.cc:654] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.175295 16418 consensus_queue.cc:260] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.189729 16451 consensus_queue.cc:1048] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.202934 16415 ts_tablet_manager.cc:1434] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.003s
I20260709 13:06:50.205024 16416 tablet_bootstrap.cc:492] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:50.205502 16453 raft_consensus.cc:493] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.205786 16416 ts_tablet_manager.cc:1403] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.185s	user 0.019s	sys 0.014s
I20260709 13:06:50.206104 16453 raft_consensus.cc:515] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.209887 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.210593 16371 raft_consensus.cc:2468] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
W20260709 13:06:50.211817 16502 log.cc:927] Time spent T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:06:50.212275 16113 leader_election.cc:304] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:50.213459 16466 raft_consensus.cc:2804] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.213891 16466 raft_consensus.cc:493] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.214223 16466 raft_consensus.cc:3060] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.215175 16418 ts_tablet_manager.cc:1434] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.173s	user 0.008s	sys 0.004s
W20260709 13:06:50.216593 16337 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 79397.9us
I20260709 13:06:50.239118 16453 leader_election.cc:290] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:50.244738 16415 tablet_bootstrap.cc:492] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:50.245255 16418 tablet_bootstrap.cc:492] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:50.251925 16416 raft_consensus.cc:359] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.252918 16416 raft_consensus.cc:385] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.253229 16416 raft_consensus.cc:740] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.254125 16416 consensus_queue.cc:260] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.256340 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:50.258705 16416 ts_tablet_manager.cc:1434] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.004s
I20260709 13:06:50.259671 16498 raft_consensus.cc:515] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.262264 16224 raft_consensus.cc:2393] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa8eb07867e7420b80a7e615d570509d in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:50.262290 16149 raft_consensus.cc:3060] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.262722 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:50.263396 16371 raft_consensus.cc:3060] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.265170 16466 raft_consensus.cc:515] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.271181 16452 raft_consensus.cc:515] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.275178 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9a1ca7eb1348438944c99fcc3eb8e9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:50.275866 16299 raft_consensus.cc:3060] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.277083 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9a1ca7eb1348438944c99fcc3eb8e9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:50.277980 16150 raft_consensus.cc:3060] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.281754 16451 consensus_queue.cc:1048] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.285097 16299 raft_consensus.cc:2468] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:50.286365 16415 tablet_bootstrap.cc:654] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.286877 16337 leader_election.cc:304] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:50.288111 16451 raft_consensus.cc:493] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.288630 16451 raft_consensus.cc:515] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.292594 16504 raft_consensus.cc:2804] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.295879 16452 leader_election.cc:290] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.296546 16504 raft_consensus.cc:697] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:50.296221 16150 raft_consensus.cc:2468] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:50.296754 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:50.301344 16445 raft_consensus.cc:493] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.302137 16445 raft_consensus.cc:515] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.302580 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:50.303422 16498 leader_election.cc:290] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.308140 16147 raft_consensus.cc:2393] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ce6da6d71124256be0679451014f776 in current term 1: Already voted for candidate aa8eb07867e7420b80a7e615d570509d in this term.
I20260709 13:06:50.311291 16451 leader_election.cc:290] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:50.312675 16504 consensus_queue.cc:237] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:06:50.313269 16114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 44822.7us
I20260709 13:06:50.313553 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:50.316166 16451 raft_consensus.cc:493] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.316877 16451 raft_consensus.cc:515] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.317749 16299 raft_consensus.cc:3060] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.318473 16264 leader_election.cc:304] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:50.319291 16451 leader_election.cc:290] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:50.316381 16416 tablet_bootstrap.cc:492] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:50.322351 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceba3482809245aa87c4e743269df937" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:50.323865 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceba3482809245aa87c4e743269df937" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
W20260709 13:06:50.323927 16262 leader_election.cc:343] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.320887 16450 raft_consensus.cc:493] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.324865 16450 raft_consensus.cc:515] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:06:50.325354 16262 leader_election.cc:343] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.325884 16262 leader_election.cc:304] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.326202 16445 leader_election.cc:290] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:50.326788 16445 raft_consensus.cc:2749] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.328037 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228d4a700de40ada51433b8043bf6a7" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:50.328564 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bd03c04594782911afd84f865c9f5" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.328770 16224 raft_consensus.cc:2468] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:50.329900 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228d4a700de40ada51433b8043bf6a7" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:50.330258 16262 leader_election.cc:304] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
W20260709 13:06:50.330389 16113 leader_election.cc:343] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.330708 16148 raft_consensus.cc:2468] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:50.331199 16451 raft_consensus.cc:2804] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.331565 16451 raft_consensus.cc:493] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.331948 16451 raft_consensus.cc:3060] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.332454 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bd03c04594782911afd84f865c9f5" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:50.335108 16114 leader_election.cc:343] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.324169 16418 tablet_bootstrap.cc:654] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.337220 16450 leader_election.cc:290] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:50.342940 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:50.343601 16371 raft_consensus.cc:2468] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:50.346093 16466 leader_election.cc:290] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:50.345799 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603bf0f99d0428194f0998267e8713d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:50.335659 16114 leader_election.cc:304] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.350550 16417 tablet_bootstrap.cc:492] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:50.350910 16188 leader_election.cc:304] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:50.351862 16510 raft_consensus.cc:2749] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.350553 16224 raft_consensus.cc:2393] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aa8eb07867e7420b80a7e615d570509d in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:50.357015 16417 ts_tablet_manager.cc:1403] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.340s	user 0.016s	sys 0.011s
I20260709 13:06:50.360016 16498 raft_consensus.cc:2804] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.360677 16498 raft_consensus.cc:697] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:50.360778 16113 leader_election.cc:304] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.362121 16510 raft_consensus.cc:2749] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.362066 16498 consensus_queue.cc:237] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.366384 16416 tablet_bootstrap.cc:654] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.370975 16415 tablet_bootstrap.cc:492] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:50.371470 16415 ts_tablet_manager.cc:1403] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.127s	user 0.010s	sys 0.001s
I20260709 13:06:50.374476 16415 raft_consensus.cc:359] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.375174 16415 raft_consensus.cc:385] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.375490 16415 raft_consensus.cc:740] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.378471 16417 raft_consensus.cc:359] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.379869 16417 raft_consensus.cc:385] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.380388 16417 raft_consensus.cc:740] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.381198 16417 consensus_queue.cc:260] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:06:50.382181 16264 consensus_peers.cc:597] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:50.387032 16453 raft_consensus.cc:493] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 115c2d35eda349178144f8365481e89c)
I20260709 13:06:50.387614 16453 raft_consensus.cc:515] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.395879 16417 ts_tablet_manager.cc:1434] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.005s
I20260709 13:06:50.398203 16417 tablet_bootstrap.cc:492] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:50.401006 16415 consensus_queue.cc:260] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.403512 16483 raft_consensus.cc:2749] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.406394 16044 catalog_manager.cc:5697] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.407284 16045 catalog_manager.cc:5697] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.413084 16504 raft_consensus.cc:493] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.413815 16504 raft_consensus.cc:515] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.417663 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd508336a7c844cc84117b7ef3a565ba" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:50.418465 16224 raft_consensus.cc:2468] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:50.420183 16500 raft_consensus.cc:493] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.420812 16500 raft_consensus.cc:515] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.423646 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.395135 16451 raft_consensus.cc:515] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.426465 16336 leader_election.cc:304] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:50.426889 16453 leader_election.cc:290] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:50.427635 16452 raft_consensus.cc:2804] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.430186 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:50.430701 16451 leader_election.cc:290] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:50.432268 16114 leader_election.cc:304] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.432989 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228d4a700de40ada51433b8043bf6a7" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:50.433610 16148 raft_consensus.cc:3060] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.434281 16417 tablet_bootstrap.cc:654] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.433739 16453 raft_consensus.cc:2749] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:50.441145 16418 tablet_bootstrap.cc:492] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:50.441715 16418 ts_tablet_manager.cc:1403] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.197s	user 0.014s	sys 0.000s
I20260709 13:06:50.428102 16452 raft_consensus.cc:493] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.444875 16452 raft_consensus.cc:3060] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.445550 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f228d4a700de40ada51433b8043bf6a7" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:50.446321 16225 raft_consensus.cc:3060] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.448076 16504 leader_election.cc:290] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.449700 16500 leader_election.cc:290] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:50.449895 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd508336a7c844cc84117b7ef3a565ba" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:50.451591 16338 leader_election.cc:343] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.456236 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:50.456683 16416 tablet_bootstrap.cc:492] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:50.457235 16416 ts_tablet_manager.cc:1403] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.141s	user 0.009s	sys 0.006s
W20260709 13:06:50.457909 16336 leader_election.cc:343] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.458796 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:50.460610 16338 leader_election.cc:343] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.461118 16338 leader_election.cc:304] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:50.462741 16500 raft_consensus.cc:2749] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.457105 16418 raft_consensus.cc:359] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.463073 16225 raft_consensus.cc:2468] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:50.462726 16148 raft_consensus.cc:2468] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:50.463704 16418 raft_consensus.cc:385] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.464265 16418 raft_consensus.cc:740] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.465161 16418 consensus_queue.cc:260] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.465499 16445 consensus_queue.cc:1048] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.468187 16416 raft_consensus.cc:359] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.468578 16264 leader_election.cc:304] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:50.468922 16416 raft_consensus.cc:385] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.469195 16416 raft_consensus.cc:740] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.469756 16483 raft_consensus.cc:2804] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.469940 16416 consensus_queue.cc:260] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.470319 16483 raft_consensus.cc:697] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:50.471159 16415 ts_tablet_manager.cc:1434] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.099s	user 0.004s	sys 0.002s
I20260709 13:06:50.472590 16418 ts_tablet_manager.cc:1434] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 13:06:50.474731 16415 tablet_bootstrap.cc:492] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:50.471361 16483 consensus_queue.cc:237] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.487234 16415 tablet_bootstrap.cc:654] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.495252 16416 ts_tablet_manager.cc:1434] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.004s
I20260709 13:06:50.496770 16416 tablet_bootstrap.cc:492] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:50.502185 16418 tablet_bootstrap.cc:492] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:50.507527 16452 raft_consensus.cc:515] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.525976 16452 leader_election.cc:290] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.522097 16418 tablet_bootstrap.cc:654] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.526983 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd508336a7c844cc84117b7ef3a565ba" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:50.527308 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd508336a7c844cc84117b7ef3a565ba" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:50.527801 16223 raft_consensus.cc:3060] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:50.528944 16338 leader_election.cc:343] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.529785 16417 tablet_bootstrap.cc:492] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:50.530228 16417 ts_tablet_manager.cc:1403] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.132s	user 0.020s	sys 0.006s
I20260709 13:06:50.541229 16223 raft_consensus.cc:2468] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:50.543232 16336 leader_election.cc:304] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:50.545549 16452 raft_consensus.cc:2804] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.546119 16452 raft_consensus.cc:697] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:50.547066 16452 consensus_queue.cc:237] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.555732 16047 catalog_manager.cc:5697] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.557444 16418 tablet_bootstrap.cc:492] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:50.557973 16418 ts_tablet_manager.cc:1403] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.072s	user 0.012s	sys 0.011s
I20260709 13:06:50.566979 16416 tablet_bootstrap.cc:654] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.568801 16418 raft_consensus.cc:359] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.569512 16418 raft_consensus.cc:385] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.569782 16418 raft_consensus.cc:740] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.570466 16418 consensus_queue.cc:260] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.555341 16417 raft_consensus.cc:359] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.574690 16417 raft_consensus.cc:385] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.575014 16417 raft_consensus.cc:740] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.582300 16418 ts_tablet_manager.cc:1434] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:06:50.583432 16417 consensus_queue.cc:260] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.587658 16417 ts_tablet_manager.cc:1434] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.001s
I20260709 13:06:50.591192 16415 tablet_bootstrap.cc:492] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:50.591714 16415 ts_tablet_manager.cc:1403] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.117s	user 0.015s	sys 0.021s
I20260709 13:06:50.593168 16417 tablet_bootstrap.cc:492] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:50.594717 16415 raft_consensus.cc:359] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.595468 16415 raft_consensus.cc:385] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.595762 16415 raft_consensus.cc:740] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.600031 16415 consensus_queue.cc:260] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.601562 16046 catalog_manager.cc:5697] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.603783 16418 tablet_bootstrap.cc:492] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:50.608191 16417 tablet_bootstrap.cc:654] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.621433 16418 tablet_bootstrap.cc:654] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.624226 16416 tablet_bootstrap.cc:492] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:50.624760 16416 ts_tablet_manager.cc:1403] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.128s	user 0.010s	sys 0.005s
I20260709 13:06:50.625082 16415 ts_tablet_manager.cc:1434] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.002s
I20260709 13:06:50.627725 16416 raft_consensus.cc:359] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.628512 16416 raft_consensus.cc:385] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.628782 16416 raft_consensus.cc:740] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.629556 16416 consensus_queue.cc:260] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.630259 16415 tablet_bootstrap.cc:492] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:50.633433 16416 ts_tablet_manager.cc:1434] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:06:50.635851 16416 tablet_bootstrap.cc:492] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:50.636663 16415 tablet_bootstrap.cc:654] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.643370 16417 tablet_bootstrap.cc:492] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:50.644124 16417 ts_tablet_manager.cc:1403] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.051s	user 0.021s	sys 0.004s
I20260709 13:06:50.653661 16416 tablet_bootstrap.cc:654] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.658774 16418 tablet_bootstrap.cc:492] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:50.659266 16418 ts_tablet_manager.cc:1403] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.076s	user 0.016s	sys 0.008s
I20260709 13:06:50.660562 16417 raft_consensus.cc:359] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.661360 16417 raft_consensus.cc:385] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.661684 16417 raft_consensus.cc:740] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.662101 16418 raft_consensus.cc:359] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.662802 16418 raft_consensus.cc:385] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.662544 16417 consensus_queue.cc:260] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.664917 16417 ts_tablet_manager.cc:1434] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.001s
I20260709 13:06:50.666002 16522 raft_consensus.cc:493] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.663089 16418 raft_consensus.cc:740] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.666567 16417 tablet_bootstrap.cc:492] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:50.667717 16418 consensus_queue.cc:260] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.666594 16522 raft_consensus.cc:515] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.671533 16522 leader_election.cc:290] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.672821 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bd03c04594782911afd84f865c9f5" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:50.673504 16148 raft_consensus.cc:2468] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:50.674553 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bd03c04594782911afd84f865c9f5" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.674973 16418 ts_tablet_manager.cc:1434] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.007s
I20260709 13:06:50.679097 16415 tablet_bootstrap.cc:492] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:50.679628 16415 ts_tablet_manager.cc:1403] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.050s	user 0.015s	sys 0.015s
I20260709 13:06:50.680002 16264 leader_election.cc:304] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
W20260709 13:06:50.681615 16262 leader_election.cc:343] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.682755 16415 raft_consensus.cc:359] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.683516 16415 raft_consensus.cc:385] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.683809 16415 raft_consensus.cc:740] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.684664 16415 consensus_queue.cc:260] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.685484 16522 raft_consensus.cc:2804] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.685871 16522 raft_consensus.cc:493] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.686203 16522 raft_consensus.cc:3060] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.687204 16415 ts_tablet_manager.cc:1434] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:50.688383 16415 tablet_bootstrap.cc:492] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:50.692449 16418 tablet_bootstrap.cc:492] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:50.695909 16416 tablet_bootstrap.cc:492] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:50.695818 16503 raft_consensus.cc:493] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.699616 16503 raft_consensus.cc:515] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.702766 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.702991 16415 tablet_bootstrap.cc:654] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.703083 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:50.703924 16148 raft_consensus.cc:2468] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
W20260709 13:06:50.704387 16188 leader_election.cc:343] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.705164 16190 leader_election.cc:304] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.706038 16498 raft_consensus.cc:2804] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.706403 16498 raft_consensus.cc:493] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.706729 16498 raft_consensus.cc:3060] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.707813 16503 leader_election.cc:290] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.708587 16522 raft_consensus.cc:515] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.709674 16452 consensus_queue.cc:1048] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:50.711404 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bd03c04594782911afd84f865c9f5" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:50.712167 16417 tablet_bootstrap.cc:654] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:50.712872 16262 leader_election.cc:343] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.714278 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bd03c04594782911afd84f865c9f5" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:50.714874 16148 raft_consensus.cc:3060] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.715729 16416 ts_tablet_manager.cc:1403] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.000s
I20260709 13:06:50.726006 16148 raft_consensus.cc:2468] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:50.725878 16416 raft_consensus.cc:359] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.728101 16416 raft_consensus.cc:385] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.715999 16522 leader_election.cc:290] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.728534 16416 raft_consensus.cc:740] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.728916 16418 tablet_bootstrap.cc:654] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.730515 16264 leader_election.cc:304] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.732628 16483 raft_consensus.cc:2804] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.733129 16483 raft_consensus.cc:697] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:50.733956 16483 consensus_queue.cc:237] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.727232 16498 raft_consensus.cc:515] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.738693 16417 tablet_bootstrap.cc:492] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:50.737191 16416 consensus_queue.cc:260] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:50.739202 16417 ts_tablet_manager.cc:1403] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.073s	user 0.013s	sys 0.004s
I20260709 13:06:50.740298 16498 leader_election.cc:290] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.741531 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:50.745903 16148 raft_consensus.cc:3060] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.743119 16503 raft_consensus.cc:493] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.743862 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:50.748380 16417 raft_consensus.cc:359] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.749151 16417 raft_consensus.cc:385] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.749432 16417 raft_consensus.cc:740] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.750295 16417 consensus_queue.cc:260] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.755147 16504 raft_consensus.cc:493] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.755343 16417 ts_tablet_manager.cc:1434] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:06:50.748468 16503 raft_consensus.cc:515] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.755724 16504 raft_consensus.cc:515] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.762502 16044 catalog_manager.cc:5697] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:50.764322 16504 leader_election.cc:290] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:50.775684 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d1691a17794035a9fae371cbabb4be" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:50.775863 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceba3482809245aa87c4e743269df937" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.758553 16503 leader_election.cc:290] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:50.765525 16417 tablet_bootstrap.cc:492] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:50.768321 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d1691a17794035a9fae371cbabb4be" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:50.776813 16299 raft_consensus.cc:2468] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:50.778245 16337 leader_election.cc:304] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:50.778646 16373 raft_consensus.cc:2468] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:50.779395 16504 raft_consensus.cc:2804] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:50.779657 16336 leader_election.cc:343] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:50.759500 16188 leader_election.cc:343] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.779876 16504 raft_consensus.cc:493] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.782434 16188 leader_election.cc:304] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:50.782526 16504 raft_consensus.cc:3060] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.783427 16503 raft_consensus.cc:2804] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:50.783885 16503 raft_consensus.cc:493] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:50.784278 16503 raft_consensus.cc:3060] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.792515 16503 raft_consensus.cc:515] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.794816 16503 leader_election.cc:290] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:50.795842 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceba3482809245aa87c4e743269df937" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:50.796085 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceba3482809245aa87c4e743269df937" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:50.796613 16299 raft_consensus.cc:3060] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.796406 16504 raft_consensus.cc:515] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.796732 16373 raft_consensus.cc:3060] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.797714 16417 tablet_bootstrap.cc:654] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.799948 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d1691a17794035a9fae371cbabb4be" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
W20260709 13:06:50.801625 16336 leader_election.cc:343] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.803998 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d1691a17794035a9fae371cbabb4be" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:50.804690 16297 raft_consensus.cc:3060] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:50.769863 16148 raft_consensus.cc:2468] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:50.744326 16416 ts_tablet_manager.cc:1434] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 13:06:50.770139 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceba3482809245aa87c4e743269df937" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:50.773798 16500 consensus_queue.cc:1048] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:50.806459 16416 tablet_bootstrap.cc:492] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:50.806725 16415 tablet_bootstrap.cc:492] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:50.807312 16415 ts_tablet_manager.cc:1403] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.026s
I20260709 13:06:50.808604 16190 leader_election.cc:304] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.810475 16415 raft_consensus.cc:359] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.811328 16415 raft_consensus.cc:385] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.812080 16503 raft_consensus.cc:2804] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.812666 16503 raft_consensus.cc:697] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:50.813568 16503 consensus_queue.cc:237] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.814662 16299 raft_consensus.cc:2468] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:50.816255 16189 leader_election.cc:304] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:50.817157 16498 raft_consensus.cc:2804] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.811659 16415 raft_consensus.cc:740] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.818279 16415 consensus_queue.cc:260] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.819279 16418 tablet_bootstrap.cc:492] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:50.819875 16418 ts_tablet_manager.cc:1403] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.141s	user 0.014s	sys 0.001s
I20260709 13:06:50.820948 16415 ts_tablet_manager.cc:1434] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:06:50.822124 16415 tablet_bootstrap.cc:492] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:50.823030 16418 raft_consensus.cc:359] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.823071 16453 raft_consensus.cc:493] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.829945 16418 raft_consensus.cc:385] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.828624 16416 tablet_bootstrap.cc:654] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.830479 16418 raft_consensus.cc:740] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.834637 16418 consensus_queue.cc:260] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.838800 16297 raft_consensus.cc:2468] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:50.832273 16504 leader_election.cc:290] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:50.842623 16418 ts_tablet_manager.cc:1434] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 13:06:50.844326 16466 raft_consensus.cc:493] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.846724 16466 raft_consensus.cc:515] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:50.832933 16453 raft_consensus.cc:515] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.850716 16466 leader_election.cc:290] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:50.850379 16453 leader_election.cc:290] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:50.852761 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3699132ec3bb4f8baa0204a17bda51f9" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:50.854293 16114 leader_election.cc:343] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.840282 16498 raft_consensus.cc:697] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:50.861981 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.863217 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:50.858399 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3699132ec3bb4f8baa0204a17bda51f9" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
W20260709 13:06:50.870674 16113 leader_election.cc:343] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:50.871448 16113 leader_election.cc:343] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.860376 16453 raft_consensus.cc:493] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.871938 16113 leader_election.cc:304] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.872099 16453 raft_consensus.cc:515] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:06:50.873629 16113 leader_election.cc:343] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.874181 16113 leader_election.cc:304] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:50.872668 16498 consensus_queue.cc:237] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.875471 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c14505e666fc47d29cfb0c9bc323384e" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:50.858151 16337 leader_election.cc:304] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776
W20260709 13:06:50.887634 16114 leader_election.cc:343] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.877763 16466 raft_consensus.cc:2749] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.887493 16418 tablet_bootstrap.cc:492] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:50.878158 16453 leader_election.cc:290] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:50.890429 16373 raft_consensus.cc:2468] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:50.892135 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c14505e666fc47d29cfb0c9bc323384e" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:50.891254 16504 raft_consensus.cc:2804] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:50.896901 16415 tablet_bootstrap.cc:654] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.900769 16507 raft_consensus.cc:2749] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.894759 16504 raft_consensus.cc:697] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
W20260709 13:06:50.893808 16113 leader_election.cc:343] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:50.906394 16417 tablet_bootstrap.cc:492] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:50.906839 16417 ts_tablet_manager.cc:1403] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.142s	user 0.015s	sys 0.006s
I20260709 13:06:50.907992 16503 consensus_queue.cc:1048] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:50.909564 16504 consensus_queue.cc:237] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.908471 16113 leader_election.cc:304] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:50.911465 16507 raft_consensus.cc:2749] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:50.915284 16417 raft_consensus.cc:359] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.916118 16417 raft_consensus.cc:385] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.916491 16417 raft_consensus.cc:740] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.917438 16417 consensus_queue.cc:260] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.921474 16417 ts_tablet_manager.cc:1434] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.003s
I20260709 13:06:50.921396 16046 catalog_manager.cc:5697] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:50.922844 16417 tablet_bootstrap.cc:492] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:50.923199 16418 tablet_bootstrap.cc:654] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.944564 16417 tablet_bootstrap.cc:654] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:50.953719 16540 consensus_queue.cc:1048] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:50.960445 16416 tablet_bootstrap.cc:492] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:50.961118 16416 ts_tablet_manager.cc:1403] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.155s	user 0.018s	sys 0.007s
I20260709 13:06:50.968910 16416 raft_consensus.cc:359] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:50.969749 16416 raft_consensus.cc:385] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.970098 16416 raft_consensus.cc:740] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.977336 16417 tablet_bootstrap.cc:492] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:50.978000 16417 ts_tablet_manager.cc:1403] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.011s
I20260709 13:06:50.981405 16417 raft_consensus.cc:359] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.982208 16417 raft_consensus.cc:385] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:50.982522 16417 raft_consensus.cc:740] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:50.986030 16498 raft_consensus.cc:493] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.986757 16498 raft_consensus.cc:515] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.988659 16417 consensus_queue.cc:260] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:50.973867 16451 raft_consensus.cc:493] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.997159 16415 tablet_bootstrap.cc:492] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:50.997822 16415 ts_tablet_manager.cc:1403] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.176s	user 0.018s	sys 0.025s
I20260709 13:06:50.999220 16452 raft_consensus.cc:493] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:50.999585 16498 leader_election.cc:290] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:50.999853 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:50.999995 16452 raft_consensus.cc:515] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.000614 16373 raft_consensus.cc:2468] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:50.993938 16416 consensus_queue.cc:260] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.006265 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.002810 16451 raft_consensus.cc:515] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.008837 16451 leader_election.cc:290] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:51.009567 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2128e5983b684efeacc628400afa861f" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.010120 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2128e5983b684efeacc628400afa861f" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.010848 16223 raft_consensus.cc:2468] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
W20260709 13:06:51.011041 16264 leader_election.cc:343] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.012044 16262 leader_election.cc:304] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:51.013134 16451 raft_consensus.cc:2804] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.013170 16415 raft_consensus.cc:359] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.014004 16415 raft_consensus.cc:385] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.014246 16415 raft_consensus.cc:740] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.013639 16451 raft_consensus.cc:493] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.015906 16451 raft_consensus.cc:3060] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.017843 16452 leader_election.cc:290] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:51.015391 16188 leader_election.cc:304] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:51.017660 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.021133 16498 raft_consensus.cc:2804] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.021569 16498 raft_consensus.cc:493] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.021809 16149 raft_consensus.cc:2393] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:51.021910 16498 raft_consensus.cc:3060] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.022213 16416 ts_tablet_manager.cc:1434] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.002s
I20260709 13:06:51.023461 16416 tablet_bootstrap.cc:492] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:51.020624 16415 consensus_queue.cc:260] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.024237 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.024747 16452 raft_consensus.cc:493] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.025270 16452 raft_consensus.cc:515] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.025761 16336 leader_election.cc:304] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
W20260709 13:06:51.020820 16190 leader_election.cc:343] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.028529 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bd03c04594782911afd84f865c9f5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:51.030072 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361bd03c04594782911afd84f865c9f5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.032042 16149 raft_consensus.cc:2393] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate 115c2d35eda349178144f8365481e89c in this term.
I20260709 13:06:51.033633 16452 leader_election.cc:290] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:51.034150 16337 leader_election.cc:304] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:51.034327 16452 raft_consensus.cc:3060] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.035336 16500 raft_consensus.cc:3060] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.035256 16498 raft_consensus.cc:515] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.038007 16417 ts_tablet_manager.cc:1434] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
I20260709 13:06:51.038522 16498 leader_election.cc:290] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:51.039201 16417 tablet_bootstrap.cc:492] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:51.028880 16044 catalog_manager.cc:5697] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.042625 16452 raft_consensus.cc:2749] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.045130 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:51.046844 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
W20260709 13:06:51.047067 16190 leader_election.cc:343] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.047511 16373 raft_consensus.cc:3060] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.054935 16415 ts_tablet_manager.cc:1434] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.057s	user 0.004s	sys 0.003s
I20260709 13:06:51.057063 16500 raft_consensus.cc:2749] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.048246 16047 catalog_manager.cc:5697] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.059212 16483 consensus_queue.cc:1048] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.060529 16418 tablet_bootstrap.cc:492] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:51.061015 16418 ts_tablet_manager.cc:1403] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.184s	user 0.026s	sys 0.009s
I20260709 13:06:51.062536 16445 consensus_queue.cc:1048] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.065433 16415 tablet_bootstrap.cc:492] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
W20260709 13:06:51.074783 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:51.080749 16415 tablet_bootstrap.cc:654] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.083977 16373 raft_consensus.cc:2468] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:51.075491 16504 raft_consensus.cc:493] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.091179 16188 leader_election.cc:304] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:51.089318 16451 raft_consensus.cc:515] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.097852 16451 leader_election.cc:290] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:51.097944 16498 raft_consensus.cc:2804] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.097251 16504 raft_consensus.cc:515] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.099807 16418 raft_consensus.cc:359] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.094702 16417 tablet_bootstrap.cc:654] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.100533 16418 raft_consensus.cc:385] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.100848 16418 raft_consensus.cc:740] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.100885 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2128e5983b684efeacc628400afa861f" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:51.099005 16498 raft_consensus.cc:697] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:51.101754 16225 raft_consensus.cc:3060] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.101652 16418 consensus_queue.cc:260] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.104136 16418 ts_tablet_manager.cc:1434] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.002s
I20260709 13:06:51.104301 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2128e5983b684efeacc628400afa861f" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:51.105110 16149 raft_consensus.cc:3060] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.105250 16418 tablet_bootstrap.cc:492] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:51.098886 16452 consensus_queue.cc:1048] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:51.109037 16338 consensus_peers.cc:597] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:51.094030 16416 tablet_bootstrap.cc:654] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.111302 16504 leader_election.cc:290] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:51.114151 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e0028829c94c99a91ca4cc4d629a4a" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.115491 16418 tablet_bootstrap.cc:654] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:51.116039 16336 leader_election.cc:343] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.107167 16498 consensus_queue.cc:237] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.130614 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e0028829c94c99a91ca4cc4d629a4a" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:51.133795 16338 leader_election.cc:343] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.134256 16338 leader_election.cc:304] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:51.135332 16504 raft_consensus.cc:2749] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.141251 16149 raft_consensus.cc:2468] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:51.147114 16540 raft_consensus.cc:493] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.147681 16540 raft_consensus.cc:515] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.141225 16225 raft_consensus.cc:2468] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:51.150619 16417 tablet_bootstrap.cc:492] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:51.151306 16417 ts_tablet_manager.cc:1403] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.112s	user 0.017s	sys 0.004s
I20260709 13:06:51.158742 16262 leader_election.cc:304] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:51.188877 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:51.189585 16445 raft_consensus.cc:2804] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.190115 16445 raft_consensus.cc:697] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
W20260709 13:06:51.190428 16189 leader_election.cc:343] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.191771 16415 tablet_bootstrap.cc:492] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:51.192304 16415 ts_tablet_manager.cc:1403] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.127s	user 0.018s	sys 0.012s
I20260709 13:06:51.193102 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:51.195184 16415 raft_consensus.cc:359] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.195896 16415 raft_consensus.cc:385] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.195766 16417 raft_consensus.cc:359] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.196272 16415 raft_consensus.cc:740] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.196758 16417 raft_consensus.cc:385] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.197324 16417 raft_consensus.cc:740] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.190868 16540 leader_election.cc:290] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:51.198813 16044 catalog_manager.cc:5697] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.195729 16445 consensus_queue.cc:237] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:06:51.202836 16188 leader_election.cc:343] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.203356 16188 leader_election.cc:304] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:51.206048 16417 consensus_queue.cc:260] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.212026 16417 ts_tablet_manager.cc:1434] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.060s	user 0.010s	sys 0.000s
I20260709 13:06:51.214781 16415 consensus_queue.cc:260] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.216933 16417 tablet_bootstrap.cc:492] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:51.219276 16416 tablet_bootstrap.cc:492] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:51.219892 16416 ts_tablet_manager.cc:1403] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.197s	user 0.009s	sys 0.012s
I20260709 13:06:51.222937 16416 raft_consensus.cc:359] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.225463 16415 ts_tablet_manager.cc:1434] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 13:06:51.227092 16417 tablet_bootstrap.cc:654] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.227684 16416 raft_consensus.cc:385] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.228029 16416 raft_consensus.cc:740] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.228679 16416 consensus_queue.cc:260] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.229660 16415 tablet_bootstrap.cc:492] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:51.214046 16540 raft_consensus.cc:2749] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.231518 16418 tablet_bootstrap.cc:492] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:51.232010 16418 ts_tablet_manager.cc:1403] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.127s	user 0.033s	sys 0.015s
I20260709 13:06:51.234604 16418 raft_consensus.cc:359] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.235237 16418 raft_consensus.cc:385] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.235498 16418 raft_consensus.cc:740] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.236814 16418 consensus_queue.cc:260] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.239169 16418 ts_tablet_manager.cc:1434] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:06:51.242112 16415 tablet_bootstrap.cc:654] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.244349 16418 tablet_bootstrap.cc:492] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:51.243302 16044 catalog_manager.cc:5697] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.261364 16416 ts_tablet_manager.cc:1434] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:06:51.259658 16445 raft_consensus.cc:493] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.270192 16540 consensus_queue.cc:1048] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.272019 16445 raft_consensus.cc:515] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.272564 16522 consensus_queue.cc:1048] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.274264 16445 leader_election.cc:290] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:51.275610 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.280231 16418 tablet_bootstrap.cc:654] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.271214 16416 tablet_bootstrap.cc:492] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:51.280148 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:51.270524 16451 raft_consensus.cc:493] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.287133 16416 tablet_bootstrap.cc:654] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.287704 16504 raft_consensus.cc:493] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.288312 16504 raft_consensus.cc:515] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.283185 16451 raft_consensus.cc:515] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.296159 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.297004 16224 raft_consensus.cc:2468] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:51.298683 16336 leader_election.cc:304] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
W20260709 13:06:51.299484 16262 leader_election.cc:343] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.299880 16552 raft_consensus.cc:2804] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.300443 16552 raft_consensus.cc:493] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.300859 16552 raft_consensus.cc:3060] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:51.301764 16262 leader_election.cc:343] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.302392 16262 leader_election.cc:304] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:51.304188 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c14505e666fc47d29cfb0c9bc323384e" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.298848 16451 leader_election.cc:290] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:51.318297 16445 raft_consensus.cc:2749] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.332489 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c14505e666fc47d29cfb0c9bc323384e" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.334983 16149 raft_consensus.cc:2468] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:51.333326 16504 leader_election.cc:290] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:51.335883 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:51.337442 16337 leader_election.cc:343] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.338073 16540 consensus_queue.cc:1048] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.341403 16264 leader_election.cc:304] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:51.342239 16451 raft_consensus.cc:2804] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.342726 16451 raft_consensus.cc:493] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.343168 16451 raft_consensus.cc:3060] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.346498 16415 tablet_bootstrap.cc:492] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
W20260709 13:06:51.346655 16262 leader_election.cc:343] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.347146 16415 ts_tablet_manager.cc:1403] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.118s	user 0.018s	sys 0.015s
I20260709 13:06:51.347954 16452 consensus_queue.cc:1048] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.350759 16552 raft_consensus.cc:515] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.351162 16415 raft_consensus.cc:359] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.353410 16415 raft_consensus.cc:385] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.353711 16415 raft_consensus.cc:740] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.355000 16417 tablet_bootstrap.cc:492] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:51.355733 16417 ts_tablet_manager.cc:1403] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.139s	user 0.005s	sys 0.020s
I20260709 13:06:51.357707 16540 consensus_queue.cc:1048] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.346397 16445 consensus_queue.cc:1048] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:06:51.373276 16418 tablet_bootstrap.cc:492] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:51.374002 16418 ts_tablet_manager.cc:1403] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.130s	user 0.040s	sys 0.024s
I20260709 13:06:51.368932 16415 consensus_queue.cc:260] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.381454 16418 raft_consensus.cc:359] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.382174 16418 raft_consensus.cc:385] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.382447 16418 raft_consensus.cc:740] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.382782 16552 leader_election.cc:290] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:51.383574 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:51.384267 16224 raft_consensus.cc:3060] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.384490 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
W20260709 13:06:51.386444 16337 leader_election.cc:343] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.383261 16418 consensus_queue.cc:260] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.393340 16418 ts_tablet_manager.cc:1434] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:06:51.391328 16451 raft_consensus.cc:515] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.408937 16417 raft_consensus.cc:359] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.409826 16417 raft_consensus.cc:385] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.410084 16417 raft_consensus.cc:740] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.411921 16416 tablet_bootstrap.cc:492] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:51.412429 16416 ts_tablet_manager.cc:1403] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.141s	user 0.017s	sys 0.006s
I20260709 13:06:51.413050 16418 tablet_bootstrap.cc:492] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
W20260709 13:06:51.415349 16336 consensus_peers.cc:597] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:51.416013 16466 raft_consensus.cc:493] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.416563 16466 raft_consensus.cc:515] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.418637 16466 leader_election.cc:290] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:51.419951 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd508336a7c844cc84117b7ef3a565ba" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:51.420557 16451 leader_election.cc:290] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:51.421272 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c14505e666fc47d29cfb0c9bc323384e" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:51.421911 16148 raft_consensus.cc:3060] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.422475 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd508336a7c844cc84117b7ef3a565ba" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.424422 16113 leader_election.cc:304] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:51.425279 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c14505e666fc47d29cfb0c9bc323384e" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:51.425319 16507 raft_consensus.cc:2749] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:51.426851 16262 leader_election.cc:343] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.431833 16415 ts_tablet_manager.cc:1434] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.014s
I20260709 13:06:51.423424 16416 raft_consensus.cc:359] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.432605 16416 raft_consensus.cc:385] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.432833 16416 raft_consensus.cc:740] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.433794 16417 consensus_queue.cc:260] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.433480 16416 consensus_queue.cc:260] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.445298 16416 ts_tablet_manager.cc:1434] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:06:51.460590 16416 tablet_bootstrap.cc:492] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:51.445628 16224 raft_consensus.cc:2468] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:51.448141 16415 tablet_bootstrap.cc:492] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:51.448385 16418 tablet_bootstrap.cc:654] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.465013 16336 leader_election.cc:304] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:51.466010 16500 raft_consensus.cc:2804] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.466559 16500 raft_consensus.cc:697] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:51.457933 16538 consensus_queue.cc:1048] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.451444 16417 ts_tablet_manager.cc:1434] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.094s	user 0.005s	sys 0.000s
I20260709 13:06:51.473729 16417 tablet_bootstrap.cc:492] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:51.465161 16148 raft_consensus.cc:2468] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:51.467499 16500 consensus_queue.cc:237] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.475952 16264 leader_election.cc:304] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:51.477306 16445 raft_consensus.cc:2804] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.477908 16445 raft_consensus.cc:697] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:51.478797 16445 consensus_queue.cc:237] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.480976 16415 tablet_bootstrap.cc:654] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.487885 16540 raft_consensus.cc:493] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.482822 16417 tablet_bootstrap.cc:654] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.486378 16416 tablet_bootstrap.cc:654] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.488826 16540 raft_consensus.cc:515] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.494354 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c337868c6fc4e079f20a204e451ab66" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.495210 16148 raft_consensus.cc:2468] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:51.495533 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c337868c6fc4e079f20a204e451ab66" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:51.496870 16190 leader_election.cc:304] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
W20260709 13:06:51.497655 16188 leader_election.cc:343] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.497908 16538 raft_consensus.cc:2804] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.498397 16538 raft_consensus.cc:493] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.498482 16540 leader_election.cc:290] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:51.498937 16538 raft_consensus.cc:3060] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.513391 16417 tablet_bootstrap.cc:492] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:51.517349 16417 ts_tablet_manager.cc:1403] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.001s
I20260709 13:06:51.525164 16538 raft_consensus.cc:515] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.543258 16416 tablet_bootstrap.cc:492] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:51.543808 16416 ts_tablet_manager.cc:1403] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.006s
I20260709 13:06:51.546836 16416 raft_consensus.cc:359] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.547614 16416 raft_consensus.cc:385] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.548044 16416 raft_consensus.cc:740] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.548988 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c337868c6fc4e079f20a204e451ab66" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:51.548795 16416 consensus_queue.cc:260] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:06:51.550683 16188 leader_election.cc:343] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.551455 16416 ts_tablet_manager.cc:1434] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:06:51.552809 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c337868c6fc4e079f20a204e451ab66" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:51.553438 16148 raft_consensus.cc:3060] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.554229 16416 tablet_bootstrap.cc:492] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:51.555856 16415 tablet_bootstrap.cc:492] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:51.556605 16415 ts_tablet_manager.cc:1403] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.109s	user 0.020s	sys 0.007s
I20260709 13:06:51.565276 16415 raft_consensus.cc:359] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.566007 16415 raft_consensus.cc:385] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.566367 16415 raft_consensus.cc:740] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.567087 16415 consensus_queue.cc:260] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.568423 16416 tablet_bootstrap.cc:654] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.565788 16418 tablet_bootstrap.cc:492] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:51.570075 16418 ts_tablet_manager.cc:1403] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.159s	user 0.028s	sys 0.005s
I20260709 13:06:51.572700 16418 raft_consensus.cc:359] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.573318 16418 raft_consensus.cc:385] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.573585 16415 ts_tablet_manager.cc:1434] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:06:51.557651 16417 raft_consensus.cc:359] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.573586 16418 raft_consensus.cc:740] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.560330 16538 leader_election.cc:290] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:51.561329 16148 raft_consensus.cc:2468] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:51.574245 16417 raft_consensus.cc:385] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.576478 16417 raft_consensus.cc:740] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.576352 16418 consensus_queue.cc:260] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.576941 16190 leader_election.cc:304] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: c8106927ed954e1c800869427b6c7964
I20260709 13:06:51.577267 16417 consensus_queue.cc:260] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.578222 16538 raft_consensus.cc:2804] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.578735 16538 raft_consensus.cc:697] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:51.579351 16417 ts_tablet_manager.cc:1434] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.061s	user 0.004s	sys 0.001s
I20260709 13:06:51.545275 16045 catalog_manager.cc:5697] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.579588 16538 consensus_queue.cc:237] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.575344 16415 tablet_bootstrap.cc:492] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:51.583819 16418 ts_tablet_manager.cc:1434] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
W20260709 13:06:51.587218 16190 consensus_peers.cc:597] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:51.591240 16418 tablet_bootstrap.cc:492] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:51.589694 16415 tablet_bootstrap.cc:654] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.600298 16418 tablet_bootstrap.cc:654] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.602334 16148 raft_consensus.cc:3060] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.605463 16044 catalog_manager.cc:5697] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.607823 16417 tablet_bootstrap.cc:492] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:51.607344 16046 catalog_manager.cc:5697] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.618393 16417 tablet_bootstrap.cc:654] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.628394 16416 tablet_bootstrap.cc:492] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:51.628901 16416 ts_tablet_manager.cc:1403] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.075s	user 0.011s	sys 0.000s
I20260709 13:06:51.636981 16522 consensus_queue.cc:1048] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.631659 16416 raft_consensus.cc:359] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.639708 16416 raft_consensus.cc:385] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.644587 16417 tablet_bootstrap.cc:492] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:51.645133 16417 ts_tablet_manager.cc:1403] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.001s
I20260709 13:06:51.649912 16418 tablet_bootstrap.cc:492] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:51.650440 16418 ts_tablet_manager.cc:1403] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.060s	user 0.008s	sys 0.002s
I20260709 13:06:51.653391 16445 consensus_queue.cc:1048] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.654066 16418 raft_consensus.cc:359] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.654719 16418 raft_consensus.cc:385] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.655004 16418 raft_consensus.cc:740] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.655819 16418 consensus_queue.cc:260] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.640009 16416 raft_consensus.cc:740] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.662413 16416 consensus_queue.cc:260] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.663511 16415 tablet_bootstrap.cc:492] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:51.664031 16415 ts_tablet_manager.cc:1403] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.089s	user 0.009s	sys 0.006s
I20260709 13:06:51.665295 16416 ts_tablet_manager.cc:1434] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:06:51.666392 16416 tablet_bootstrap.cc:492] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:51.666774 16415 raft_consensus.cc:359] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.667474 16415 raft_consensus.cc:385] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.667758 16415 raft_consensus.cc:740] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.669083 16418 ts_tablet_manager.cc:1434] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:06:51.670292 16418 tablet_bootstrap.cc:492] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:51.672678 16522 raft_consensus.cc:493] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.673213 16522 raft_consensus.cc:515] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.673142 16417 raft_consensus.cc:359] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.674073 16417 raft_consensus.cc:385] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.674335 16417 raft_consensus.cc:740] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.674998 16417 consensus_queue.cc:260] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.675436 16522 leader_election.cc:290] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:51.672814 16416 tablet_bootstrap.cc:654] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.677385 16417 ts_tablet_manager.cc:1434] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.002s
I20260709 13:06:51.679155 16507 raft_consensus.cc:493] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.679546 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e652cb454b674e399c2ac22c22c05506" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.679677 16507 raft_consensus.cc:515] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:06:51.682310 16262 leader_election.cc:343] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.672060 16415 consensus_queue.cc:260] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.684301 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.684510 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:51.685001 16225 raft_consensus.cc:2468] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:51.685209 16297 raft_consensus.cc:2468] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:51.684547 16417 tablet_bootstrap.cc:492] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:51.685873 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e652cb454b674e399c2ac22c22c05506" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:51.687050 16416 tablet_bootstrap.cc:492] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:51.687510 16416 ts_tablet_manager.cc:1403] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.008s
I20260709 13:06:51.689466 16466 raft_consensus.cc:493] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.689916 16466 raft_consensus.cc:515] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.690169 16416 raft_consensus.cc:359] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.690789 16416 raft_consensus.cc:385] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.691051 16416 raft_consensus.cc:740] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.691663 16416 consensus_queue.cc:260] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.692076 16466 leader_election.cc:290] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:51.694336 16417 tablet_bootstrap.cc:654] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.694458 16416 ts_tablet_manager.cc:1434] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:51.695513 16416 tablet_bootstrap.cc:492] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
W20260709 13:06:51.697374 16262 leader_election.cc:343] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.697862 16262 leader_election.cc:304] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:51.699818 16113 leader_election.cc:304] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:51.701570 16416 tablet_bootstrap.cc:654] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.703284 16538 consensus_queue.cc:1048] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.704419 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.704460 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:51.707188 16114 leader_election.cc:304] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:51.708551 16466 raft_consensus.cc:2804] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.709003 16466 raft_consensus.cc:493] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.709398 16466 raft_consensus.cc:3060] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.715890 16418 tablet_bootstrap.cc:654] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.717320 16507 leader_election.cc:290] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:51.718194 16522 raft_consensus.cc:2749] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.718598 16507 raft_consensus.cc:2749] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.720122 16415 ts_tablet_manager.cc:1434] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 13:06:51.721606 16415 tablet_bootstrap.cc:492] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:51.731165 16415 tablet_bootstrap.cc:654] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.741025 16416 tablet_bootstrap.cc:492] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:51.764249 16416 ts_tablet_manager.cc:1403] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.002s
I20260709 13:06:51.756294 16417 tablet_bootstrap.cc:492] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:51.738895 16445 raft_consensus.cc:493] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.765602 16417 ts_tablet_manager.cc:1403] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.011s
I20260709 13:06:51.765668 16445 raft_consensus.cc:515] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.767894 16538 raft_consensus.cc:493] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.768409 16538 raft_consensus.cc:515] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.768846 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:51.769320 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.768435 16417 raft_consensus.cc:359] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.770414 16417 raft_consensus.cc:385] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.770471 16538 leader_election.cc:290] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:06:51.770876 16264 leader_election.cc:343] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.770787 16417 raft_consensus.cc:740] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.772229 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e0028829c94c99a91ca4cc4d629a4a" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:51.772436 16417 consensus_queue.cc:260] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.773321 16415 tablet_bootstrap.cc:492] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:51.773587 16538 raft_consensus.cc:493] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:51.773249 16262 leader_election.cc:343] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.774088 16538 raft_consensus.cc:515] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.774480 16415 ts_tablet_manager.cc:1403] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.000s
I20260709 13:06:51.774806 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e0028829c94c99a91ca4cc4d629a4a" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.745473 16466 raft_consensus.cc:515] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:06:51.776328 16190 leader_election.cc:343] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.774616 16417 ts_tablet_manager.cc:1434] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:06:51.777278 16538 leader_election.cc:290] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:51.772888 16373 raft_consensus.cc:2468] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:51.778519 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:51.779276 16225 raft_consensus.cc:3060] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.779452 16188 leader_election.cc:304] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:51.774503 16262 leader_election.cc:304] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:06:51.778469 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c14505e666fc47d29cfb0c9bc323384e" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:51.780686 16417 tablet_bootstrap.cc:492] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:51.780835 16522 raft_consensus.cc:2749] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.781121 16149 raft_consensus.cc:3060] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.771045 16445 leader_election.cc:290] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:51.767078 16416 raft_consensus.cc:359] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.784302 16466 leader_election.cc:290] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:51.786036 16416 raft_consensus.cc:385] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.786382 16416 raft_consensus.cc:740] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.786870 16225 raft_consensus.cc:2468] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:51.787079 16416 consensus_queue.cc:260] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.788102 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:51.788767 16297 raft_consensus.cc:3060] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.789234 16416 ts_tablet_manager.cc:1434] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:06:51.790247 16416 tablet_bootstrap.cc:492] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:51.788185 16540 raft_consensus.cc:2804] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.791903 16540 raft_consensus.cc:493] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.792286 16540 raft_consensus.cc:3060] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.793027 16417 tablet_bootstrap.cc:654] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.781138 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c14505e666fc47d29cfb0c9bc323384e" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.796675 16148 raft_consensus.cc:2393] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce6da6d71124256be0679451014f776 in current term 1: Already voted for candidate 115c2d35eda349178144f8365481e89c in this term.
I20260709 13:06:51.796995 16415 raft_consensus.cc:359] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.797668 16415 raft_consensus.cc:385] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.797950 16415 raft_consensus.cc:740] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.798154 16113 leader_election.cc:304] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:51.798609 16415 consensus_queue.cc:260] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.799000 16466 raft_consensus.cc:2804] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.799499 16466 raft_consensus.cc:697] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:51.800326 16466 consensus_queue.cc:237] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:51.804791 16190 leader_election.cc:304] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:51.805884 16503 raft_consensus.cc:3060] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.812348 16417 tablet_bootstrap.cc:492] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:51.814736 16417 ts_tablet_manager.cc:1403] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.006s
I20260709 13:06:51.778488 16418 tablet_bootstrap.cc:492] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:51.815797 16418 ts_tablet_manager.cc:1403] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.146s	user 0.008s	sys 0.008s
I20260709 13:06:51.818116 16417 raft_consensus.cc:359] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.819348 16417 raft_consensus.cc:385] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.819715 16417 raft_consensus.cc:740] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.818812 16503 raft_consensus.cc:2749] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.820734 16540 raft_consensus.cc:515] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.821118 16418 raft_consensus.cc:359] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.821758 16418 raft_consensus.cc:385] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.822031 16418 raft_consensus.cc:740] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.822881 16540 leader_election.cc:290] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:51.824172 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e0028829c94c99a91ca4cc4d629a4a" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:51.824877 16370 raft_consensus.cc:3060] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.825399 16415 ts_tablet_manager.cc:1434] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.000s
I20260709 13:06:51.825495 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e0028829c94c99a91ca4cc4d629a4a" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
W20260709 13:06:51.827322 16190 leader_election.cc:343] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.828168 16297 raft_consensus.cc:2468] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:51.830193 16500 raft_consensus.cc:493] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.830708 16500 raft_consensus.cc:515] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.832134 16416 tablet_bootstrap.cc:654] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.832856 16500 leader_election.cc:290] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:51.833652 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:51.834738 16370 raft_consensus.cc:2468] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
W20260709 13:06:51.835613 16337 leader_election.cc:343] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.836211 16522 raft_consensus.cc:493] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.837409 16522 raft_consensus.cc:515] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.838068 16188 leader_election.cc:304] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:51.839421 16503 raft_consensus.cc:2804] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.839668 16522 leader_election.cc:290] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:51.840036 16503 raft_consensus.cc:697] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:51.840641 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.840335 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb9eaed164d46ccac187f1878f24e4a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.841845 16225 raft_consensus.cc:2468] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:51.841413 16503 consensus_queue.cc:237] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.836020 16417 consensus_queue.cc:260] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.842739 16418 consensus_queue.cc:260] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.843658 16500 raft_consensus.cc:493] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:06:51.844041 16338 leader_election.cc:343] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.844254 16500 raft_consensus.cc:515] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.845619 16418 ts_tablet_manager.cc:1434] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:06:51.843540 16262 leader_election.cc:304] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:51.846642 16418 tablet_bootstrap.cc:492] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:51.847326 16445 raft_consensus.cc:2804] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.847724 16445 raft_consensus.cc:493] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.848095 16445 raft_consensus.cc:3060] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.849601 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e652cb454b674e399c2ac22c22c05506" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.850310 16225 raft_consensus.cc:2468] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:51.851866 16336 leader_election.cc:304] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:51.844520 16338 leader_election.cc:304] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:51.852777 16504 raft_consensus.cc:2804] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.853183 16504 raft_consensus.cc:493] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.853225 16452 raft_consensus.cc:2749] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:51.853632 16504 raft_consensus.cc:3060] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.859107 16418 tablet_bootstrap.cc:654] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.861892 16504 raft_consensus.cc:515] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.863914 16416 tablet_bootstrap.cc:492] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:51.864393 16416 ts_tablet_manager.cc:1403] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.074s	user 0.014s	sys 0.000s
I20260709 13:06:51.867044 16416 raft_consensus.cc:359] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.867676 16416 raft_consensus.cc:385] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:51.867956 16416 raft_consensus.cc:740] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:51.868000 16452 consensus_queue.cc:1048] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:51.870462 16416 consensus_queue.cc:260] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.871706 16500 leader_election.cc:290] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:51.876536 16416 ts_tablet_manager.cc:1434] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:06:51.876653 16415 tablet_bootstrap.cc:492] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:51.880939 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb9eaed164d46ccac187f1878f24e4a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:51.872882 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e652cb454b674e399c2ac22c22c05506" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:51.886286 16297 raft_consensus.cc:2468] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:51.886871 16046 catalog_manager.cc:5697] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:51.894548 16417 ts_tablet_manager.cc:1434] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.003s
I20260709 13:06:51.897774 16500 raft_consensus.cc:493] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.898276 16500 raft_consensus.cc:515] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.898540 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e652cb454b674e399c2ac22c22c05506" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:51.898833 16416 tablet_bootstrap.cc:492] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:51.899147 16225 raft_consensus.cc:3060] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.895478 16504 leader_election.cc:290] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:51.901635 16500 leader_election.cc:290] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:51.901886 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3699132ec3bb4f8baa0204a17bda51f9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.902709 16148 raft_consensus.cc:2468] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
W20260709 13:06:51.903188 16262 leader_election.cc:343] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.904779 16338 leader_election.cc:304] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:51.905818 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e652cb454b674e399c2ac22c22c05506" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:51.906090 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3699132ec3bb4f8baa0204a17bda51f9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:51.906426 16297 raft_consensus.cc:3060] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.906688 16299 raft_consensus.cc:2468] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:51.910421 16504 raft_consensus.cc:2804] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:51.911755 16504 raft_consensus.cc:493] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:51.912107 16504 raft_consensus.cc:3060] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.913261 16415 tablet_bootstrap.cc:654] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.903492 16500 raft_consensus.cc:493] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:51.896906 16445 raft_consensus.cc:515] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.930539 16297 raft_consensus.cc:2468] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:51.931910 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb9eaed164d46ccac187f1878f24e4a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
W20260709 13:06:51.933224 16337 consensus_peers.cc:597] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:51.935494 16416 tablet_bootstrap.cc:654] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:51.938422 16417 tablet_bootstrap.cc:492] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:51.944123 16451 consensus_queue.cc:1048] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:51.946082 16337 leader_election.cc:304] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:51.947047 16552 raft_consensus.cc:2804] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:51.947526 16552 raft_consensus.cc:697] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:51.948361 16552 consensus_queue.cc:237] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:51.948511 16044 catalog_manager.cc:5697] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:51.918378 16500 raft_consensus.cc:515] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:51.940431 16223 raft_consensus.cc:3060] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.967372 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb9eaed164d46ccac187f1878f24e4a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
W20260709 13:06:51.969957 16262 leader_election.cc:343] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:51.970392 16225 raft_consensus.cc:2468] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:51.961941 16445 leader_election.cc:290] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:51.974817 16500 leader_election.cc:290] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:51.938231 16504 raft_consensus.cc:515] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:51.976934 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c337868c6fc4e079f20a204e451ab66" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:51.977869 16148 raft_consensus.cc:2393] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:51.980280 16504 leader_election.cc:290] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:51.981467 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3699132ec3bb4f8baa0204a17bda51f9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:51.982422 16148 raft_consensus.cc:3060] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:51.985963 16418 tablet_bootstrap.cc:492] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:51.986496 16418 ts_tablet_manager.cc:1403] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.140s	user 0.005s	sys 0.025s
I20260709 13:06:51.993664 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c337868c6fc4e079f20a204e451ab66" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:51.995222 16336 leader_election.cc:304] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:52.000241 16417 tablet_bootstrap.cc:654] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.005450 16504 raft_consensus.cc:3060] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.010131 16418 raft_consensus.cc:359] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.012239 16223 raft_consensus.cc:2468] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:52.010933 16418 raft_consensus.cc:385] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.013640 16418 raft_consensus.cc:740] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.013945 16262 leader_election.cc:304] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: c8106927ed954e1c800869427b6c7964
I20260709 13:06:52.014576 16418 consensus_queue.cc:260] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.015460 16522 raft_consensus.cc:2804] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.016430 16522 raft_consensus.cc:697] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:52.017601 16522 consensus_queue.cc:237] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.018244 16418 ts_tablet_manager.cc:1434] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 13:06:52.023259 16148 raft_consensus.cc:2468] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:51.993662 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3699132ec3bb4f8baa0204a17bda51f9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:52.025415 16338 leader_election.cc:304] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:52.024806 16297 raft_consensus.cc:3060] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.027504 16504 raft_consensus.cc:2749] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.032136 16415 tablet_bootstrap.cc:492] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:52.032342 16416 tablet_bootstrap.cc:492] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:52.032647 16415 ts_tablet_manager.cc:1403] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.160s	user 0.016s	sys 0.013s
I20260709 13:06:52.032852 16416 ts_tablet_manager.cc:1403] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.134s	user 0.029s	sys 0.020s
I20260709 13:06:52.029834 16500 raft_consensus.cc:2804] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.030455 16452 raft_consensus.cc:493] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.036101 16452 raft_consensus.cc:515] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.039505 16451 consensus_queue.cc:1048] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.040735 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:52.041676 16299 raft_consensus.cc:2468] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:52.042165 16415 raft_consensus.cc:359] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.042773 16415 raft_consensus.cc:385] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.042949 16415 raft_consensus.cc:740] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.043409 16415 consensus_queue.cc:260] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.041945 16500 raft_consensus.cc:697] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:52.045178 16452 leader_election.cc:290] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:52.046669 16500 consensus_queue.cc:237] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.048293 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:52.048645 16540 consensus_queue.cc:1048] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:06:52.054301 16223 raft_consensus.cc:2468] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:52.054280 16337 leader_election.cc:304] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:52.055501 16044 catalog_manager.cc:5697] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:52.059414 16452 raft_consensus.cc:2804] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.060132 16452 raft_consensus.cc:493] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.060519 16452 raft_consensus.cc:3060] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.064067 16415 ts_tablet_manager.cc:1434] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:06:52.069362 16418 tablet_bootstrap.cc:492] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:52.077934 16417 tablet_bootstrap.cc:492] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:52.078521 16417 ts_tablet_manager.cc:1403] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.161s	user 0.009s	sys 0.014s
I20260709 13:06:52.083544 16417 raft_consensus.cc:359] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.084246 16417 raft_consensus.cc:385] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.084514 16417 raft_consensus.cc:740] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.085712 16417 consensus_queue.cc:260] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.087859 16415 tablet_bootstrap.cc:492] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:52.091123 16417 ts_tablet_manager.cc:1434] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.005s
I20260709 13:06:52.102442 16045 catalog_manager.cc:5697] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.106760 16417 tablet_bootstrap.cc:492] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:52.112268 16498 consensus_queue.cc:1048] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.113806 16417 tablet_bootstrap.cc:654] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.119912 16416 raft_consensus.cc:359] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.120958 16416 raft_consensus.cc:385] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.121317 16416 raft_consensus.cc:740] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.122526 16297 raft_consensus.cc:2468] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:52.122361 16416 consensus_queue.cc:260] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.127228 16416 ts_tablet_manager.cc:1434] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.002s
I20260709 13:06:52.128857 16416 tablet_bootstrap.cc:492] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:52.130967 16418 tablet_bootstrap.cc:654] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.134572 16044 catalog_manager.cc:5697] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:52.136971 16416 tablet_bootstrap.cc:654] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.153036 16418 tablet_bootstrap.cc:492] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:52.153569 16418 ts_tablet_manager.cc:1403] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.116s	user 0.006s	sys 0.007s
I20260709 13:06:52.159471 16452 raft_consensus.cc:515] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.163828 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:52.164485 16223 raft_consensus.cc:3060] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.184713 16415 tablet_bootstrap.cc:654] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.187937 16452 leader_election.cc:290] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:52.193722 16507 raft_consensus.cc:493] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.198176 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:52.201095 16297 raft_consensus.cc:3060] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.201246 16507 raft_consensus.cc:515] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.206815 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:52.207448 16417 tablet_bootstrap.cc:492] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:52.207991 16417 ts_tablet_manager.cc:1403] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.109s	user 0.046s	sys 0.018s
W20260709 13:06:52.208340 16113 leader_election.cc:343] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.208240 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:52.211669 16416 tablet_bootstrap.cc:492] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
W20260709 13:06:52.212167 16113 leader_election.cc:343] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.216298 16416 ts_tablet_manager.cc:1403] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.088s	user 0.017s	sys 0.019s
I20260709 13:06:52.216168 16113 leader_election.cc:304] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:52.217106 16466 raft_consensus.cc:2749] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.218446 16297 raft_consensus.cc:2468] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:52.219614 16337 leader_election.cc:304] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:52.219719 16416 raft_consensus.cc:359] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.220465 16452 raft_consensus.cc:2804] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.220628 16416 raft_consensus.cc:385] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.220958 16452 raft_consensus.cc:697] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:52.221057 16416 raft_consensus.cc:740] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.222416 16466 raft_consensus.cc:493] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.221954 16452 consensus_queue.cc:237] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
W20260709 13:06:52.222869 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:52.222926 16466 raft_consensus.cc:515] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.209713 16507 leader_election.cc:290] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:52.212929 16223 raft_consensus.cc:2468] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:52.225489 16498 raft_consensus.cc:493] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.226013 16498 raft_consensus.cc:515] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.214483 16417 raft_consensus.cc:359] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.227207 16417 raft_consensus.cc:385] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.227967 16522 raft_consensus.cc:493] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.228456 16522 raft_consensus.cc:515] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.230029 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e0028829c94c99a91ca4cc4d629a4a" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:52.230597 16522 leader_election.cc:290] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:52.231031 16370 raft_consensus.cc:2393] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa8eb07867e7420b80a7e615d570509d in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:52.227523 16417 raft_consensus.cc:740] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.225241 16466 leader_election.cc:290] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:52.235802 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b521a909fd464b72bc2742ff3157658b" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:52.237591 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b521a909fd464b72bc2742ff3157658b" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
W20260709 13:06:52.240059 16262 leader_election.cc:343] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:52.240954 16262 leader_election.cc:343] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.241482 16262 leader_election.cc:304] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:52.232012 16417 consensus_queue.cc:260] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.242262 16445 raft_consensus.cc:2749] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.243506 16417 ts_tablet_manager.cc:1434] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 13:06:52.240006 16418 raft_consensus.cc:359] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.243920 16418 raft_consensus.cc:385] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.244210 16418 raft_consensus.cc:740] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.244588 16417 tablet_bootstrap.cc:492] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:52.245023 16418 consensus_queue.cc:260] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.260872 16466 consensus_queue.cc:1048] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.221995 16416 consensus_queue.cc:260] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.279616 16552 raft_consensus.cc:493] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.280323 16552 raft_consensus.cc:515] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.282037 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47e0028829c94c99a91ca4cc4d629a4a" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:52.295915 16113 leader_election.cc:304] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:52.296981 16507 raft_consensus.cc:3060] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.284662 16466 raft_consensus.cc:493] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.285133 16503 raft_consensus.cc:493] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.297910 16503 raft_consensus.cc:515] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.300136 16466 raft_consensus.cc:515] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.364288 16418 ts_tablet_manager.cc:1434] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.210s	user 0.010s	sys 0.019s
I20260709 13:06:52.364804 16416 ts_tablet_manager.cc:1434] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.148s	user 0.018s	sys 0.012s
I20260709 13:06:52.365810 16416 tablet_bootstrap.cc:492] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:52.366530 16415 tablet_bootstrap.cc:492] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:52.368850 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7973b570c8554e048bd5a9d02fd13a46" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:52.370177 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:52.371867 16418 tablet_bootstrap.cc:492] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:52.372180 16298 raft_consensus.cc:2468] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
W20260709 13:06:52.373178 16338 leader_election.cc:343] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.373684 16114 leader_election.cc:304] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
W20260709 13:06:52.373733 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 132800us
I20260709 13:06:52.374704 16589 raft_consensus.cc:2804] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.375202 16589 raft_consensus.cc:493] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.375599 16589 raft_consensus.cc:3060] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.378479 16418 tablet_bootstrap.cc:654] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.384460 16415 ts_tablet_manager.cc:1403] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.298s	user 0.018s	sys 0.013s
I20260709 13:06:52.371214 16466 leader_election.cc:290] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
W20260709 13:06:52.388773 16336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 155269us
I20260709 13:06:52.385258 16552 leader_election.cc:290] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:52.395320 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d1691a17794035a9fae371cbabb4be" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:52.397037 16417 tablet_bootstrap.cc:654] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.407546 16503 leader_election.cc:290] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:52.416982 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:52.410295 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:52.421777 16044 catalog_manager.cc:5697] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:52.423554 16188 leader_election.cc:343] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.423892 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d1691a17794035a9fae371cbabb4be" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:52.426115 16416 tablet_bootstrap.cc:654] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.392971 16589 raft_consensus.cc:515] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.427397 16498 leader_election.cc:290] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:52.429584 16466 consensus_queue.cc:1048] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.435896 16415 raft_consensus.cc:359] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.436627 16415 raft_consensus.cc:385] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.436905 16415 raft_consensus.cc:740] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.437939 16589 leader_election.cc:290] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:52.438910 16188 leader_election.cc:304] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:52.439810 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:52.440672 16148 raft_consensus.cc:2468] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:52.431923 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7973b570c8554e048bd5a9d02fd13a46" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:52.441107 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:52.441702 16298 raft_consensus.cc:3060] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.442116 16190 leader_election.cc:304] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: c8106927ed954e1c800869427b6c7964
W20260709 13:06:52.442332 16336 leader_election.cc:343] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.442823 16336 leader_election.cc:304] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:52.444042 16504 raft_consensus.cc:2749] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.449031 16415 consensus_queue.cc:260] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.440989 16498 raft_consensus.cc:2749] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.457677 16503 raft_consensus.cc:2804] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.458127 16503 raft_consensus.cc:493] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:52.458263 16415 ts_tablet_manager.cc:1434] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.001s
I20260709 13:06:52.458477 16503 raft_consensus.cc:3060] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.463167 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:52.472514 16452 raft_consensus.cc:493] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.489311 16418 tablet_bootstrap.cc:492] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:52.489780 16418 ts_tablet_manager.cc:1403] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.124s	user 0.015s	sys 0.030s
I20260709 13:06:52.494274 16507 raft_consensus.cc:2749] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.495097 16415 tablet_bootstrap.cc:492] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:52.484054 16298 raft_consensus.cc:2468] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:52.497864 16538 consensus_queue.cc:1048] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.498646 16418 raft_consensus.cc:359] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.499338 16418 raft_consensus.cc:385] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.499607 16418 raft_consensus.cc:740] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.500315 16418 consensus_queue.cc:260] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.502599 16418 ts_tablet_manager.cc:1434] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:06:52.504746 16114 leader_election.cc:304] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
W20260709 13:06:52.483067 16113 leader_election.cc:343] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.506260 16418 tablet_bootstrap.cc:492] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:52.506636 16507 raft_consensus.cc:2804] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.509321 16507 raft_consensus.cc:697] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:52.509483 16416 tablet_bootstrap.cc:492] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:52.509980 16416 ts_tablet_manager.cc:1403] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.144s	user 0.009s	sys 0.012s
I20260709 13:06:52.510166 16507 consensus_queue.cc:237] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.480535 16452 raft_consensus.cc:515] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.512524 16225 raft_consensus.cc:3060] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.514185 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb9eaed164d46ccac187f1878f24e4a" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:52.514609 16500 consensus_queue.cc:1048] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.515205 16224 raft_consensus.cc:2393] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate 115c2d35eda349178144f8365481e89c in this term.
I20260709 13:06:52.539474 16417 tablet_bootstrap.cc:492] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:52.539253 16500 consensus_queue.cc:1048] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:52.540063 16417 ts_tablet_manager.cc:1403] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.296s	user 0.037s	sys 0.034s
I20260709 13:06:52.542455 16498 consensus_queue.cc:1048] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.542873 16417 raft_consensus.cc:359] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.543532 16417 raft_consensus.cc:385] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.544003 16417 raft_consensus.cc:740] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.550388 16417 consensus_queue.cc:260] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.568416 16552 consensus_queue.cc:1048] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.571951 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffb9eaed164d46ccac187f1878f24e4a" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:52.572556 16503 raft_consensus.cc:515] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.574093 16337 leader_election.cc:304] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:52.574728 16503 leader_election.cc:290] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:52.575620 16522 consensus_queue.cc:1048] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:52.526681 16113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 44681us
W20260709 13:06:52.579017 16113 leader_election.cc:343] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.583891 16417 ts_tablet_manager.cc:1434] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:06:52.595114 16500 raft_consensus.cc:493] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.595674 16500 raft_consensus.cc:515] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.582549 16452 leader_election.cc:290] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:52.600133 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:52.600692 16500 leader_election.cc:290] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:52.600948 16371 raft_consensus.cc:3060] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.602043 16417 tablet_bootstrap.cc:492] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:52.585273 16552 raft_consensus.cc:3060] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.604282 16415 tablet_bootstrap.cc:654] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.600135 16416 raft_consensus.cc:359] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.615262 16416 raft_consensus.cc:385] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.616320 16416 raft_consensus.cc:740] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.618655 16046 catalog_manager.cc:5697] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:52.608495 16618 raft_consensus.cc:493] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.611526 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:52.628284 16416 consensus_queue.cc:260] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.632092 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:52.633142 16221 raft_consensus.cc:2393] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:52.640497 16416 ts_tablet_manager.cc:1434] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.130s	user 0.008s	sys 0.000s
I20260709 13:06:52.641522 16552 raft_consensus.cc:2749] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.626878 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:52.636054 16150 raft_consensus.cc:3060] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.629745 16618 raft_consensus.cc:515] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.644655 16417 tablet_bootstrap.cc:654] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.646315 16618 leader_election.cc:290] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:52.669226 16418 tablet_bootstrap.cc:654] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.671969 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
W20260709 13:06:52.674836 16623 log.cc:927] Time spent T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:06:52.678805 16150 raft_consensus.cc:2468] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:52.690521 16416 tablet_bootstrap.cc:492] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:52.707727 16190 leader_election.cc:304] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:52.711081 16498 raft_consensus.cc:2804] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:52.665756 16371 raft_consensus.cc:2468] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:52.711622 16498 raft_consensus.cc:697] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:52.712612 16498 consensus_queue.cc:237] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.685928 16338 leader_election.cc:304] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:52.728415 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:52.745438 16552 raft_consensus.cc:2749] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:52.746299 16338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 66628.4us
I20260709 13:06:52.749495 16445 consensus_queue.cc:1048] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.761433 16417 tablet_bootstrap.cc:492] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:52.762059 16417 ts_tablet_manager.cc:1403] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.160s	user 0.028s	sys 0.006s
I20260709 13:06:52.769593 16416 tablet_bootstrap.cc:654] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.771674 16113 leader_election.cc:304] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:52.772576 16618 raft_consensus.cc:2749] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.774439 16417 raft_consensus.cc:359] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.775182 16417 raft_consensus.cc:385] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.775449 16417 raft_consensus.cc:740] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.776356 16417 consensus_queue.cc:260] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:52.778724 16417 ts_tablet_manager.cc:1434] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:06:52.778927 16452 consensus_queue.cc:1048] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.779811 16417 tablet_bootstrap.cc:492] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:52.784932 16418 tablet_bootstrap.cc:492] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:52.785640 16418 ts_tablet_manager.cc:1403] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.282s	user 0.020s	sys 0.013s
I20260709 13:06:52.786206 16415 tablet_bootstrap.cc:492] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:52.795846 16552 consensus_queue.cc:1048] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:52.799825 16415 ts_tablet_manager.cc:1403] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.310s	user 0.008s	sys 0.035s
I20260709 13:06:52.804065 16452 consensus_queue.cc:1048] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:06:52.816156 16418 raft_consensus.cc:359] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.816933 16418 raft_consensus.cc:385] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.817219 16418 raft_consensus.cc:740] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.818900 16418 consensus_queue.cc:260] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.821116 16418 ts_tablet_manager.cc:1434] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.004s
I20260709 13:06:52.824913 16150 raft_consensus.cc:3060] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.829602 16418 tablet_bootstrap.cc:492] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:52.845317 16418 tablet_bootstrap.cc:654] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.845965 16046 catalog_manager.cc:5697] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:52.850695 16415 raft_consensus.cc:359] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.855086 16415 raft_consensus.cc:385] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.860937 16445 raft_consensus.cc:493] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.861523 16445 raft_consensus.cc:515] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.855612 16415 raft_consensus.cc:740] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.863029 16415 consensus_queue.cc:260] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.865063 16445 leader_election.cc:290] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:52.867336 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:52.877472 16416 tablet_bootstrap.cc:492] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:52.878080 16416 ts_tablet_manager.cc:1403] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.198s	user 0.012s	sys 0.001s
I20260709 13:06:52.869845 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:52.874708 16417 tablet_bootstrap.cc:654] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:52.878706 16264 leader_election.cc:343] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.881049 16415 ts_tablet_manager.cc:1434] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.001s
I20260709 13:06:52.882109 16415 tablet_bootstrap.cc:492] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
W20260709 13:06:52.885234 16262 leader_election.cc:343] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.885754 16262 leader_election.cc:304] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:06:52.886785 16445 raft_consensus.cc:2749] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.889251 16416 raft_consensus.cc:359] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.890944 16416 raft_consensus.cc:385] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.891686 16416 raft_consensus.cc:740] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.892894 16416 consensus_queue.cc:260] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:52.897923 16416 ts_tablet_manager.cc:1434] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:06:52.899963 16416 tablet_bootstrap.cc:492] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:52.900768 16415 tablet_bootstrap.cc:654] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.904953 16418 tablet_bootstrap.cc:492] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:52.905495 16418 ts_tablet_manager.cc:1403] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.001s
I20260709 13:06:52.908509 16418 raft_consensus.cc:359] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.909197 16418 raft_consensus.cc:385] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.909477 16418 raft_consensus.cc:740] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.907012 16417 tablet_bootstrap.cc:492] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:52.910166 16418 consensus_queue.cc:260] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.911585 16417 ts_tablet_manager.cc:1403] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.131s	user 0.013s	sys 0.000s
I20260709 13:06:52.914567 16417 raft_consensus.cc:359] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.915252 16417 raft_consensus.cc:385] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.915545 16417 raft_consensus.cc:740] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.916350 16417 consensus_queue.cc:260] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.920186 16416 tablet_bootstrap.cc:654] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.923676 16417 ts_tablet_manager.cc:1434] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:06:52.924783 16417 tablet_bootstrap.cc:492] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:52.929961 16638 raft_consensus.cc:493] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.930514 16638 raft_consensus.cc:515] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.932775 16638 leader_election.cc:290] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:52.933993 16418 ts_tablet_manager.cc:1434] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:06:52.934625 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:52.935480 16298 raft_consensus.cc:2393] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate aa8eb07867e7420b80a7e615d570509d in this term.
I20260709 13:06:52.938597 16418 tablet_bootstrap.cc:492] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:52.938546 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:52.943233 16415 tablet_bootstrap.cc:492] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:52.943812 16415 ts_tablet_manager.cc:1403] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.002s
I20260709 13:06:52.944480 16338 leader_election.cc:304] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:52.946952 16638 raft_consensus.cc:3060] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.945241 16418 tablet_bootstrap.cc:654] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.958216 16415 raft_consensus.cc:359] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.959293 16415 raft_consensus.cc:385] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.959645 16415 raft_consensus.cc:740] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.964531 16417 tablet_bootstrap.cc:654] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:52.969682 16638 raft_consensus.cc:2749] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:52.974349 16417 tablet_bootstrap.cc:492] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:52.974874 16417 ts_tablet_manager.cc:1403] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.050s	user 0.008s	sys 0.002s
I20260709 13:06:52.977768 16417 raft_consensus.cc:359] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.978458 16417 raft_consensus.cc:385] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.978556 16416 tablet_bootstrap.cc:492] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:52.978761 16417 raft_consensus.cc:740] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.979344 16416 ts_tablet_manager.cc:1403] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.015s
I20260709 13:06:52.979664 16417 consensus_queue.cc:260] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:52.982224 16416 raft_consensus.cc:359] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.982973 16416 raft_consensus.cc:385] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:52.983290 16416 raft_consensus.cc:740] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:52.984512 16416 consensus_queue.cc:260] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.986663 16415 consensus_queue.cc:260] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.988759 16415 ts_tablet_manager.cc:1434] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 13:06:52.988912 16498 raft_consensus.cc:493] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:52.989387 16498 raft_consensus.cc:515] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:52.991379 16498 leader_election.cc:290] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:52.991714 16418 tablet_bootstrap.cc:492] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:52.992270 16418 ts_tablet_manager.cc:1403] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.007s
I20260709 13:06:52.992944 16416 ts_tablet_manager.cc:1434] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:06:52.994165 16416 tablet_bootstrap.cc:492] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:52.995031 16415 tablet_bootstrap.cc:492] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:52.992905 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76774e4153c7483683fcec5f557bfaad" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:52.995800 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76774e4153c7483683fcec5f557bfaad" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:52.996467 16150 raft_consensus.cc:2468] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
W20260709 13:06:52.996577 16189 leader_election.cc:343] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:52.996907 16417 ts_tablet_manager.cc:1434] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:06:52.997762 16190 leader_election.cc:304] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:52.998947 16498 raft_consensus.cc:2804] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:52.999948 16498 raft_consensus.cc:493] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.000310 16498 raft_consensus.cc:3060] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.001060 16503 raft_consensus.cc:493] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.001585 16503 raft_consensus.cc:515] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.003758 16503 leader_election.cc:290] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:53.005558 16416 tablet_bootstrap.cc:654] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.012238 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9db6747820949b6b76dcc3ab05485b1" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.012482 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9db6747820949b6b76dcc3ab05485b1" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:53.013049 16298 raft_consensus.cc:2468] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
W20260709 13:06:53.013988 16190 leader_election.cc:343] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.014578 16189 leader_election.cc:304] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:53.015875 16503 raft_consensus.cc:2804] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.016240 16503 raft_consensus.cc:493] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.016604 16503 raft_consensus.cc:3060] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:52.995082 16418 raft_consensus.cc:359] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.025058 16418 raft_consensus.cc:385] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.025377 16418 raft_consensus.cc:740] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.027809 16416 tablet_bootstrap.cc:492] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:53.028448 16416 ts_tablet_manager.cc:1403] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.004s
I20260709 13:06:53.029676 16498 raft_consensus.cc:515] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.031498 16416 raft_consensus.cc:359] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.032383 16416 raft_consensus.cc:385] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.032557 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76774e4153c7483683fcec5f557bfaad" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:53.033169 16150 raft_consensus.cc:3060] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.032744 16416 raft_consensus.cc:740] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.033608 16417 tablet_bootstrap.cc:492] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:53.034197 16416 consensus_queue.cc:260] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.036573 16416 ts_tablet_manager.cc:1434] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:53.037724 16416 tablet_bootstrap.cc:492] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:53.047346 16416 tablet_bootstrap.cc:654] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.049659 16503 raft_consensus.cc:515] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.052786 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9db6747820949b6b76dcc3ab05485b1" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
W20260709 13:06:53.054510 16190 leader_election.cc:343] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.055198 16150 raft_consensus.cc:2468] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:53.056962 16190 leader_election.cc:304] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:53.058102 16538 raft_consensus.cc:2804] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.058749 16538 raft_consensus.cc:697] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:53.059999 16417 tablet_bootstrap.cc:654] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.060170 16538 consensus_queue.cc:237] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.065409 16415 tablet_bootstrap.cc:654] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.053318 16418 consensus_queue.cc:260] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.069193 16503 leader_election.cc:290] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:53.074450 16417 tablet_bootstrap.cc:492] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:53.075026 16417 ts_tablet_manager.cc:1403] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.042s	user 0.005s	sys 0.005s
I20260709 13:06:53.076514 16618 consensus_queue.cc:1048] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.079849 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9db6747820949b6b76dcc3ab05485b1" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:53.080725 16298 raft_consensus.cc:3060] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.083446 16417 raft_consensus.cc:359] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.084282 16417 raft_consensus.cc:385] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.084550 16417 raft_consensus.cc:740] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.087170 16415 tablet_bootstrap.cc:492] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:53.087708 16415 ts_tablet_manager.cc:1403] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.000s
I20260709 13:06:53.090709 16415 raft_consensus.cc:359] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.092867 16503 raft_consensus.cc:493] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.107930 16503 raft_consensus.cc:515] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.105480 16507 consensus_queue.cc:1048] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.110128 16503 leader_election.cc:290] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:53.106456 16445 raft_consensus.cc:493] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.111430 16445 raft_consensus.cc:515] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.113019 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7186244ab72e4b32ad068941b913f052" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:53.114174 16445 leader_election.cc:290] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
W20260709 13:06:53.115414 16188 leader_election.cc:343] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.116781 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2201023e16254eed9712e5fba24caa38" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:06:53.118734 16262 leader_election.cc:343] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.122960 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7186244ab72e4b32ad068941b913f052" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:53.124576 16189 leader_election.cc:343] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.125047 16189 leader_election.cc:304] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:53.132309 16538 raft_consensus.cc:2749] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.136507 16538 consensus_queue.cc:1048] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.145056 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2201023e16254eed9712e5fba24caa38" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:53.148137 16264 leader_election.cc:343] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.148700 16264 leader_election.cc:304] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:53.164358 16604 raft_consensus.cc:2749] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.166810 16415 raft_consensus.cc:385] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.167193 16415 raft_consensus.cc:740] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.168277 16415 consensus_queue.cc:260] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.085491 16417 consensus_queue.cc:260] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.173272 16415 ts_tablet_manager.cc:1434] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.085s	user 0.005s	sys 0.001s
I20260709 13:06:53.174533 16415 tablet_bootstrap.cc:492] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:53.179075 16417 ts_tablet_manager.cc:1434] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.104s	user 0.004s	sys 0.004s
I20260709 13:06:53.181344 16417 tablet_bootstrap.cc:492] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:53.183979 16418 ts_tablet_manager.cc:1434] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.191s	user 0.007s	sys 0.001s
I20260709 13:06:53.185532 16538 consensus_queue.cc:1048] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.193714 16418 tablet_bootstrap.cc:492] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:53.195246 16047 catalog_manager.cc:5697] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.197453 16498 leader_election.cc:290] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:53.222827 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76774e4153c7483683fcec5f557bfaad" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:53.223649 16416 tablet_bootstrap.cc:492] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:53.232240 16416 ts_tablet_manager.cc:1403] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.195s	user 0.061s	sys 0.068s
W20260709 13:06:53.234225 16189 leader_election.cc:343] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.235069 16416 raft_consensus.cc:359] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.235980 16416 raft_consensus.cc:385] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.236307 16416 raft_consensus.cc:740] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.236840 16415 tablet_bootstrap.cc:654] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.227360 16417 tablet_bootstrap.cc:654] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.224748 16418 tablet_bootstrap.cc:654] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.237275 16416 consensus_queue.cc:260] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.247800 16638 raft_consensus.cc:493] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.248499 16638 raft_consensus.cc:515] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.250182 16416 ts_tablet_manager.cc:1434] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:06:53.257755 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:06:53.259294 16336 leader_election.cc:343] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.260510 16298 raft_consensus.cc:2468] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:53.261864 16189 leader_election.cc:304] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:53.268265 16416 tablet_bootstrap.cc:492] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:53.264608 16498 raft_consensus.cc:2804] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.273789 16498 raft_consensus.cc:697] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:53.274677 16498 consensus_queue.cc:237] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.264816 16638 leader_election.cc:290] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:53.282313 16416 tablet_bootstrap.cc:654] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.284605 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:53.285367 16150 raft_consensus.cc:2468] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:53.285622 16417 tablet_bootstrap.cc:492] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:53.286178 16417 ts_tablet_manager.cc:1403] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.009s
I20260709 13:06:53.287098 16338 leader_election.cc:304] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:53.296495 16500 raft_consensus.cc:2804] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.297072 16500 raft_consensus.cc:493] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.297545 16500 raft_consensus.cc:3060] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.303449 16417 raft_consensus.cc:359] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.304288 16417 raft_consensus.cc:385] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.304559 16417 raft_consensus.cc:740] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.305172 16417 consensus_queue.cc:260] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.310837 16417 ts_tablet_manager.cc:1434] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:06:53.311964 16417 tablet_bootstrap.cc:492] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:53.314054 16540 raft_consensus.cc:493] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.314635 16540 raft_consensus.cc:515] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.315047 16507 raft_consensus.cc:493] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.315547 16507 raft_consensus.cc:515] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.317574 16507 leader_election.cc:290] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:53.317582 16540 leader_election.cc:290] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:53.319597 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.323339 16417 tablet_bootstrap.cc:654] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:53.327461 16114 leader_election.cc:343] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.328217 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b521a909fd464b72bc2742ff3157658b" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.328876 16298 raft_consensus.cc:2468] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:53.330652 16500 raft_consensus.cc:515] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.332067 16189 leader_election.cc:304] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:53.333415 16540 raft_consensus.cc:2804] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.333806 16540 raft_consensus.cc:493] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.334123 16540 raft_consensus.cc:3060] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.336126 16500 leader_election.cc:290] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:53.336381 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:53.336817 16418 tablet_bootstrap.cc:492] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:53.336688 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b521a909fd464b72bc2742ff3157658b" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:53.337330 16418 ts_tablet_manager.cc:1403] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.144s	user 0.020s	sys 0.000s
W20260709 13:06:53.337990 16336 leader_election.cc:343] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:53.338217 16188 leader_election.cc:343] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.340122 16418 raft_consensus.cc:359] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.340034 16045 catalog_manager.cc:5697] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:53.341817 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:53.340809 16418 raft_consensus.cc:385] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.343154 16418 raft_consensus.cc:740] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.343971 16418 consensus_queue.cc:260] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.344619 16415 tablet_bootstrap.cc:492] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:53.345103 16415 ts_tablet_manager.cc:1403] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.171s	user 0.025s	sys 0.007s
I20260709 13:06:53.345669 16418 ts_tablet_manager.cc:1434] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:06:53.346637 16418 tablet_bootstrap.cc:492] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:53.349812 16415 raft_consensus.cc:359] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.350546 16415 raft_consensus.cc:385] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.350811 16415 raft_consensus.cc:740] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.351446 16415 consensus_queue.cc:260] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.353513 16415 ts_tablet_manager.cc:1434] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:06:53.353922 16418 tablet_bootstrap.cc:654] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.358474 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:53.359241 16372 raft_consensus.cc:2468] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:53.361274 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:53.361841 16150 raft_consensus.cc:3060] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.361428 16113 leader_election.cc:304] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:53.362641 16507 raft_consensus.cc:2804] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.363092 16507 raft_consensus.cc:493] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.363482 16507 raft_consensus.cc:3060] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.372143 16150 raft_consensus.cc:2468] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:53.373270 16338 leader_election.cc:304] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:53.374899 16500 raft_consensus.cc:2804] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.376003 16500 raft_consensus.cc:697] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:53.377209 16500 consensus_queue.cc:237] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.381441 16540 raft_consensus.cc:515] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.384667 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b521a909fd464b72bc2742ff3157658b" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:53.385486 16299 raft_consensus.cc:3060] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.386299 16538 raft_consensus.cc:493] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.386798 16538 raft_consensus.cc:515] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.388854 16418 tablet_bootstrap.cc:492] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:53.389046 16538 leader_election.cc:290] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:53.389621 16418 ts_tablet_manager.cc:1403] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.043s	user 0.005s	sys 0.011s
I20260709 13:06:53.390895 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62345643f7684ec08f60f740de27cead" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.391736 16298 raft_consensus.cc:2468] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:53.393354 16507 raft_consensus.cc:515] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.394434 16189 leader_election.cc:304] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:53.395623 16538 raft_consensus.cc:2804] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.396112 16538 raft_consensus.cc:493] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.396512 16538 raft_consensus.cc:3060] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.396883 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
W20260709 13:06:53.398254 16114 leader_election.cc:343] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.401903 16415 tablet_bootstrap.cc:492] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:53.390947 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62345643f7684ec08f60f740de27cead" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:53.405470 16190 leader_election.cc:343] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.410157 16299 raft_consensus.cc:2468] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:53.411880 16189 leader_election.cc:304] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:53.412709 16498 raft_consensus.cc:2804] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.413281 16498 raft_consensus.cc:697] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:53.414101 16498 consensus_queue.cc:237] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.414885 16540 leader_election.cc:290] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:53.416419 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b521a909fd464b72bc2742ff3157658b" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:53.417430 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
W20260709 13:06:53.417850 16188 leader_election.cc:343] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.418052 16371 raft_consensus.cc:3060] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.418362 16538 raft_consensus.cc:515] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.421240 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62345643f7684ec08f60f740de27cead" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:53.421732 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62345643f7684ec08f60f740de27cead" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:53.422384 16298 raft_consensus.cc:3060] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:53.422585 16190 leader_election.cc:343] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.426695 16371 raft_consensus.cc:2468] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:53.427819 16416 tablet_bootstrap.cc:492] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:53.429148 16415 tablet_bootstrap.cc:654] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.429498 16416 ts_tablet_manager.cc:1403] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.163s	user 0.016s	sys 0.013s
I20260709 13:06:53.428249 16538 leader_election.cc:290] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:53.432044 16418 raft_consensus.cc:359] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.432649 16418 raft_consensus.cc:385] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.432933 16418 raft_consensus.cc:740] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.428380 16507 leader_election.cc:290] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:53.435474 16418 consensus_queue.cc:260] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.442812 16113 leader_election.cc:304] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:53.443801 16418 ts_tablet_manager.cc:1434] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 13:06:53.446297 16298 raft_consensus.cc:2468] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:53.446516 16452 raft_consensus.cc:493] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.451996 16452 raft_consensus.cc:515] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.452745 16416 raft_consensus.cc:359] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.453390 16417 tablet_bootstrap.cc:492] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:53.453521 16416 raft_consensus.cc:385] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.453898 16416 raft_consensus.cc:740] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.453123 16045 catalog_manager.cc:5697] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.453984 16417 ts_tablet_manager.cc:1403] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.142s	user 0.016s	sys 0.014s
I20260709 13:06:53.454614 16416 consensus_queue.cc:260] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.455209 16618 raft_consensus.cc:2804] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.455837 16618 raft_consensus.cc:697] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:53.456393 16418 tablet_bootstrap.cc:492] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:53.456524 16589 raft_consensus.cc:493] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.456776 16618 consensus_queue.cc:237] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.457206 16589 raft_consensus.cc:515] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.457728 16417 raft_consensus.cc:359] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.458784 16417 raft_consensus.cc:385] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.459041 16417 raft_consensus.cc:740] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.466979 16416 ts_tablet_manager.cc:1434] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:06:53.468560 16416 tablet_bootstrap.cc:492] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:53.473891 16418 tablet_bootstrap.cc:654] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.483083 16417 consensus_queue.cc:260] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.486624 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.484257 16189 leader_election.cc:304] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:53.489298 16503 raft_consensus.cc:2804] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.489949 16503 raft_consensus.cc:697] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
W20260709 13:06:53.490154 16337 leader_election.cc:343] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.517895 16048 catalog_manager.cc:5697] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.528882 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.529698 16298 raft_consensus.cc:2468] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:53.528887 16452 leader_election.cc:290] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:53.538542 16417 ts_tablet_manager.cc:1434] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 13:06:53.491222 16503 consensus_queue.cc:237] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.545748 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:53.523504 16538 raft_consensus.cc:493] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.548988 16417 tablet_bootstrap.cc:492] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:53.549252 16452 raft_consensus.cc:493] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.549908 16452 raft_consensus.cc:515] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.552012 16538 raft_consensus.cc:515] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.548123 16045 catalog_manager.cc:5697] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:53.524181 16589 leader_election.cc:290] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:53.541256 16114 leader_election.cc:304] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:53.557456 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:53.558348 16418 tablet_bootstrap.cc:492] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:53.558499 16507 raft_consensus.cc:2804] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.558880 16418 ts_tablet_manager.cc:1403] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.103s	user 0.034s	sys 0.026s
I20260709 13:06:53.559031 16538 leader_election.cc:290] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:06:53.559310 16113 leader_election.cc:343] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.559367 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:53.560245 16149 raft_consensus.cc:2468] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:53.561861 16418 raft_consensus.cc:359] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.562562 16418 raft_consensus.cc:385] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.562821 16418 raft_consensus.cc:740] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.562943 16190 leader_election.cc:304] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
W20260709 13:06:53.564054 16338 leader_election.cc:343] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.564383 16498 raft_consensus.cc:2804] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.564767 16498 raft_consensus.cc:493] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.565102 16498 raft_consensus.cc:3060] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.567096 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:53.558947 16507 raft_consensus.cc:493] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.568231 16507 raft_consensus.cc:3060] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.569754 16417 tablet_bootstrap.cc:654] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.572016 16372 raft_consensus.cc:2468] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:53.573769 16416 tablet_bootstrap.cc:654] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.563527 16418 consensus_queue.cc:260] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.576272 16338 leader_election.cc:304] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:53.577368 16638 raft_consensus.cc:2749] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.580108 16418 ts_tablet_manager.cc:1434] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:06:53.580732 16452 leader_election.cc:290] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:53.581236 16418 tablet_bootstrap.cc:492] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:53.605279 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:53.605957 16149 raft_consensus.cc:2468] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:53.608036 16338 leader_election.cc:304] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:53.609261 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:53.610069 16452 raft_consensus.cc:2804] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.610464 16452 raft_consensus.cc:493] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.610585 16418 tablet_bootstrap.cc:654] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.610867 16452 raft_consensus.cc:3060] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.615123 16604 raft_consensus.cc:493] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.615844 16604 raft_consensus.cc:515] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.619712 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cec4e308d104a9f9049fa0c2263b58d" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:06:53.626246 16262 leader_election.cc:343] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.630241 16604 leader_election.cc:290] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:53.639976 16452 raft_consensus.cc:515] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.646682 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:53.647289 16149 raft_consensus.cc:3060] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.635677 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cec4e308d104a9f9049fa0c2263b58d" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
W20260709 13:06:53.650364 16262 leader_election.cc:343] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.650805 16262 leader_election.cc:304] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:53.651881 16604 raft_consensus.cc:2749] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.624375 16498 raft_consensus.cc:515] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.654575 16498 leader_election.cc:290] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:06:53.654898 16189 consensus_peers.cc:597] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:53.654894 16538 consensus_queue.cc:1048] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.656584 16221 raft_consensus.cc:2393] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:53.656955 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:53.657574 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:53.658092 16371 raft_consensus.cc:2393] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ce6da6d71124256be0679451014f776 in current term 1: Already voted for candidate c8106927ed954e1c800869427b6c7964 in this term.
I20260709 13:06:53.659991 16500 raft_consensus.cc:493] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.660630 16500 raft_consensus.cc:515] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.647186 16507 raft_consensus.cc:515] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.664741 16417 tablet_bootstrap.cc:492] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:53.665252 16417 ts_tablet_manager.cc:1403] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.117s	user 0.021s	sys 0.007s
I20260709 13:06:53.667852 16417 raft_consensus.cc:359] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.668481 16417 raft_consensus.cc:385] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.668721 16417 raft_consensus.cc:740] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.671236 16452 leader_election.cc:290] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:53.672694 16417 consensus_queue.cc:260] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.674242 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:53.674971 16221 raft_consensus.cc:2393] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:53.678069 16417 ts_tablet_manager.cc:1434] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:06:53.678893 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e24ed51cb74d0da9f8b77dcbdab5b9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.679327 16417 tablet_bootstrap.cc:492] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
W20260709 13:06:53.680511 16337 leader_election.cc:343] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.681016 16188 leader_election.cc:304] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:06:53.681879 16415 tablet_bootstrap.cc:492] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:53.682332 16415 ts_tablet_manager.cc:1403] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.281s	user 0.032s	sys 0.058s
I20260709 13:06:53.682489 16538 raft_consensus.cc:2749] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.671770 16500 leader_election.cc:290] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:53.685015 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e24ed51cb74d0da9f8b77dcbdab5b9" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:53.686331 16149 raft_consensus.cc:2468] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:53.691534 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:53.691887 16507 leader_election.cc:290] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:53.692307 16299 raft_consensus.cc:3060] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.691926 16416 tablet_bootstrap.cc:492] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:53.696278 16416 ts_tablet_manager.cc:1403] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.228s	user 0.010s	sys 0.029s
I20260709 13:06:53.697718 16418 tablet_bootstrap.cc:492] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:53.698019 16417 tablet_bootstrap.cc:654] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.698211 16418 ts_tablet_manager.cc:1403] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.117s	user 0.026s	sys 0.008s
I20260709 13:06:53.699074 16416 raft_consensus.cc:359] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.699836 16416 raft_consensus.cc:385] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.694921 16048 catalog_manager.cc:5697] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.700213 16416 raft_consensus.cc:740] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
W20260709 13:06:53.702760 16338 leader_election.cc:343] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.705245 16338 leader_election.cc:304] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:53.704852 16299 raft_consensus.cc:2468] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:53.708628 16114 leader_election.cc:304] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:53.709910 16418 raft_consensus.cc:359] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.711210 16418 raft_consensus.cc:385] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.711439 16418 raft_consensus.cc:740] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.712766 16418 consensus_queue.cc:260] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.711836 16507 raft_consensus.cc:2804] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.715152 16507 raft_consensus.cc:697] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:53.716184 16507 consensus_queue.cc:237] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.714321 16416 consensus_queue.cc:260] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.711974 16500 raft_consensus.cc:2749] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.719669 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:53.723378 16418 ts_tablet_manager.cc:1434] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
I20260709 13:06:53.724574 16415 raft_consensus.cc:359] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.725596 16415 raft_consensus.cc:385] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.725929 16415 raft_consensus.cc:740] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.726670 16415 consensus_queue.cc:260] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.722048 16338 leader_election.cc:304] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:53.732257 16500 raft_consensus.cc:2804] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.733829 16500 raft_consensus.cc:697] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:53.734658 16500 consensus_queue.cc:237] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:53.737129 16417 tablet_bootstrap.cc:492] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
W20260709 13:06:53.732653 16113 leader_election.cc:343] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.739225 16415 ts_tablet_manager.cc:1434] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.003s
I20260709 13:06:53.740372 16415 tablet_bootstrap.cc:492] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:53.728268 16416 ts_tablet_manager.cc:1434] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:06:53.746487 16416 tablet_bootstrap.cc:492] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:53.746706 16415 tablet_bootstrap.cc:654] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.737656 16417 ts_tablet_manager.cc:1403] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.005s
I20260709 13:06:53.755729 16452 raft_consensus.cc:493] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.756419 16452 raft_consensus.cc:515] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.759109 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7186244ab72e4b32ad068941b913f052" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:53.759765 16221 raft_consensus.cc:2468] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:53.760907 16336 leader_election.cc:304] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:53.761684 16638 raft_consensus.cc:2804] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.762058 16638 raft_consensus.cc:493] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.762413 16638 raft_consensus.cc:3060] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.764137 16619 raft_consensus.cc:493] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.764673 16619 raft_consensus.cc:515] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.766887 16619 leader_election.cc:290] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:53.768971 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9db6747820949b6b76dcc3ab05485b1" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.770102 16299 raft_consensus.cc:2393] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa8eb07867e7420b80a7e615d570509d in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:53.770260 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9db6747820949b6b76dcc3ab05485b1" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:53.775141 16114 leader_election.cc:304] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:53.775884 16418 tablet_bootstrap.cc:492] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:53.778374 16619 raft_consensus.cc:3060] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.780009 16452 leader_election.cc:290] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:53.781049 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7186244ab72e4b32ad068941b913f052" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.781872 16638 raft_consensus.cc:515] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.804041 16417 raft_consensus.cc:359] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.805698 16417 raft_consensus.cc:385] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.807408 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7186244ab72e4b32ad068941b913f052" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:53.808207 16221 raft_consensus.cc:3060] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.806075 16417 raft_consensus.cc:740] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.812099 16638 leader_election.cc:290] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:53.811932 16048 catalog_manager.cc:5697] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:53.812901 16337 leader_election.cc:343] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.818315 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7186244ab72e4b32ad068941b913f052" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:53.812188 16417 consensus_queue.cc:260] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:06:53.834098 16337 leader_election.cc:343] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:53.819816 16336 consensus_peers.cc:597] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:53.841257 16299 raft_consensus.cc:3060] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:53.846448 16188 consensus_peers.cc:597] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 -> Peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Couldn't send request to peer c8106927ed954e1c800869427b6c7964. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:53.862233 16417 ts_tablet_manager.cc:1434] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.000s
I20260709 13:06:53.862461 16415 tablet_bootstrap.cc:492] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:53.863988 16415 ts_tablet_manager.cc:1403] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.124s	user 0.029s	sys 0.061s
I20260709 13:06:53.864271 16416 tablet_bootstrap.cc:654] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.872613 16415 raft_consensus.cc:359] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.873456 16415 raft_consensus.cc:385] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:53.873795 16415 raft_consensus.cc:740] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:53.874493 16415 consensus_queue.cc:260] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.876698 16415 ts_tablet_manager.cc:1434] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:06:53.877146 16221 raft_consensus.cc:2468] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:53.878307 16415 tablet_bootstrap.cc:492] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:53.881383 16619 raft_consensus.cc:2749] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:53.886933 16417 tablet_bootstrap.cc:492] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:53.889387 16619 consensus_queue.cc:1048] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:53.892814 16336 leader_election.cc:304] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:53.893898 16415 tablet_bootstrap.cc:654] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.894083 16504 raft_consensus.cc:2804] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:53.894790 16504 raft_consensus.cc:697] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:53.895934 16504 consensus_queue.cc:237] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:53.898990 16417 tablet_bootstrap.cc:654] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:53.905339 16503 consensus_queue.cc:1048] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.910542 16540 consensus_queue.cc:1048] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.923074 16503 consensus_queue.cc:1048] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:53.925659 16445 raft_consensus.cc:493] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.926203 16445 raft_consensus.cc:515] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.918286 16048 catalog_manager.cc:5697] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:53.948432 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:53.949469 16372 raft_consensus.cc:2468] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:53.940153 16638 raft_consensus.cc:493] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.958670 16262 leader_election.cc:304] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:53.962252 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:53.962944 16149 raft_consensus.cc:2468] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:53.963944 16451 raft_consensus.cc:2804] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:53.958953 16638 raft_consensus.cc:515] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.964347 16451 raft_consensus.cc:493] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:53.964742 16451 raft_consensus.cc:3060] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.965986 16638 leader_election.cc:290] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:53.969786 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:53.970311 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.971115 16296 raft_consensus.cc:2393] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate aa8eb07867e7420b80a7e615d570509d in this term.
I20260709 13:06:53.973524 16338 leader_election.cc:304] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:53.975199 16500 raft_consensus.cc:3060] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:53.974478 16416 tablet_bootstrap.cc:492] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:53.977368 16416 ts_tablet_manager.cc:1403] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.231s	user 0.014s	sys 0.006s
I20260709 13:06:53.978541 16503 raft_consensus.cc:493] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:53.979028 16503 raft_consensus.cc:515] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:53.981133 16503 leader_election.cc:290] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:53.982666 16451 raft_consensus.cc:515] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:53.984164 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd94eae6c1241e5838930e00eb43c4c" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:53.987188 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd94eae6c1241e5838930e00eb43c4c" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:53.988219 16189 leader_election.cc:343] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.955063 16445 leader_election.cc:290] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:53.985889 16417 tablet_bootstrap.cc:492] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:53.990917 16417 ts_tablet_manager.cc:1403] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.112s	user 0.010s	sys 0.009s
W20260709 13:06:53.990487 16190 leader_election.cc:343] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:53.996001 16190 leader_election.cc:304] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:53.997653 16540 raft_consensus.cc:2749] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.000805 16416 raft_consensus.cc:359] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.001567 16416 raft_consensus.cc:385] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.001842 16416 raft_consensus.cc:740] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.002599 16416 consensus_queue.cc:260] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.004701 16149 raft_consensus.cc:3060] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.014598 16417 raft_consensus.cc:359] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.016528 16417 raft_consensus.cc:385] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.016814 16417 raft_consensus.cc:740] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.017268 16416 ts_tablet_manager.cc:1434] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
I20260709 13:06:54.018751 16418 tablet_bootstrap.cc:654] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.018831 16504 consensus_queue.cc:1048] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:54.029013 16114 consensus_peers.cc:597] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.017473 16417 consensus_queue.cc:260] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.055873 16415 tablet_bootstrap.cc:492] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:54.056370 16415 ts_tablet_manager.cc:1403] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.179s	user 0.014s	sys 0.007s
I20260709 13:06:54.056399 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:54.057245 16372 raft_consensus.cc:3060] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.059037 16415 raft_consensus.cc:359] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.059731 16415 raft_consensus.cc:385] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.060418 16415 raft_consensus.cc:740] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.061223 16415 consensus_queue.cc:260] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.062039 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:54.063052 16147 raft_consensus.cc:3060] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.064455 16415 ts_tablet_manager.cc:1434] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:06:54.065634 16415 tablet_bootstrap.cc:492] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:54.066099 16416 tablet_bootstrap.cc:492] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:54.072127 16451 leader_election.cc:290] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:54.080318 16417 ts_tablet_manager.cc:1434] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.089s	user 0.007s	sys 0.001s
I20260709 13:06:54.083477 16500 raft_consensus.cc:2749] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.086026 16147 raft_consensus.cc:2468] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:54.086984 16415 tablet_bootstrap.cc:654] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.081699 16418 tablet_bootstrap.cc:492] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:54.092875 16418 ts_tablet_manager.cc:1403] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.366s	user 0.016s	sys 0.002s
I20260709 13:06:54.093293 16417 tablet_bootstrap.cc:492] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:54.095228 16507 raft_consensus.cc:493] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.095758 16507 raft_consensus.cc:515] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.098423 16540 consensus_queue.cc:1048] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.099987 16638 raft_consensus.cc:493] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.100476 16638 raft_consensus.cc:515] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.116276 16638 leader_election.cc:290] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:54.117865 16372 raft_consensus.cc:2468] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:54.117800 16264 leader_election.cc:304] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:54.118672 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62345643f7684ec08f60f740de27cead" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.119163 16451 raft_consensus.cc:2804] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.119575 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cec4e308d104a9f9049fa0c2263b58d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.119910 16451 raft_consensus.cc:697] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:54.122008 16507 leader_election.cc:290] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:54.123018 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cec4e308d104a9f9049fa0c2263b58d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:54.124375 16299 raft_consensus.cc:2468] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:54.123576 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62345643f7684ec08f60f740de27cead" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:54.126024 16337 leader_election.cc:304] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:54.128863 16638 raft_consensus.cc:2804] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:54.129249 16336 leader_election.cc:343] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.130002 16113 leader_election.cc:304] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:54.130789 16589 consensus_queue.cc:1048] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.131033 16618 raft_consensus.cc:2749] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:54.120977 16451 consensus_queue.cc:237] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.129280 16638 raft_consensus.cc:493] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.132536 16638 raft_consensus.cc:3060] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.127946 16418 raft_consensus.cc:359] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.137387 16418 raft_consensus.cc:385] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.137709 16418 raft_consensus.cc:740] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.139720 16498 consensus_queue.cc:1048] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.144652 16418 consensus_queue.cc:260] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.148947 16417 tablet_bootstrap.cc:654] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.159404 16418 ts_tablet_manager.cc:1434] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 13:06:54.159107 16045 catalog_manager.cc:5697] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.167085 16416 tablet_bootstrap.cc:654] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.169834 16415 tablet_bootstrap.cc:492] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:54.170389 16415 ts_tablet_manager.cc:1403] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.105s	user 0.004s	sys 0.010s
I20260709 13:06:54.173218 16415 raft_consensus.cc:359] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.173892 16415 raft_consensus.cc:385] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.174146 16415 raft_consensus.cc:740] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.174826 16415 consensus_queue.cc:260] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.176600 16415 ts_tablet_manager.cc:1434] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:06:54.177575 16415 tablet_bootstrap.cc:492] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:54.188442 16415 tablet_bootstrap.cc:654] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.200016 16418 tablet_bootstrap.cc:492] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:54.214164 16638 raft_consensus.cc:515] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.229331 16418 tablet_bootstrap.cc:654] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.268005 16417 tablet_bootstrap.cc:492] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:54.267726 16046 catalog_manager.cc:5697] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.268770 16417 ts_tablet_manager.cc:1403] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.180s	user 0.046s	sys 0.028s
I20260709 13:06:54.272352 16417 raft_consensus.cc:359] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.273231 16417 raft_consensus.cc:385] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.273563 16417 raft_consensus.cc:740] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.274379 16417 consensus_queue.cc:260] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.277799 16417 ts_tablet_manager.cc:1434] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:06:54.279110 16417 tablet_bootstrap.cc:492] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:54.293658 16417 tablet_bootstrap.cc:654] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.296556 16418 tablet_bootstrap.cc:492] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:54.297109 16418 ts_tablet_manager.cc:1403] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.099s	user 0.011s	sys 0.003s
I20260709 13:06:54.297684 16638 leader_election.cc:290] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:54.301534 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cec4e308d104a9f9049fa0c2263b58d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:54.304190 16418 raft_consensus.cc:359] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.304935 16418 raft_consensus.cc:385] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:06:54.308449 16336 leader_election.cc:343] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.303668 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cec4e308d104a9f9049fa0c2263b58d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:54.309474 16298 raft_consensus.cc:3060] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.311437 16619 consensus_queue.cc:1048] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:06:54.314855 16337 consensus_peers.cc:597] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:54.316736 16504 consensus_queue.cc:1048] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.317564 16415 tablet_bootstrap.cc:492] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:54.318017 16415 ts_tablet_manager.cc:1403] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.141s	user 0.025s	sys 0.033s
I20260709 13:06:54.320622 16415 raft_consensus.cc:359] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.321250 16415 raft_consensus.cc:385] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.321501 16415 raft_consensus.cc:740] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.322103 16415 consensus_queue.cc:260] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.306406 16418 raft_consensus.cc:740] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.325115 16418 consensus_queue.cc:260] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.327823 16415 ts_tablet_manager.cc:1434] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:06:54.328912 16415 tablet_bootstrap.cc:492] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:54.331086 16451 raft_consensus.cc:493] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.331588 16451 raft_consensus.cc:515] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.333735 16451 leader_election.cc:290] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:54.334542 16415 tablet_bootstrap.cc:654] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.335670 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:54.336174 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:54.337824 16262 leader_election.cc:304] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:06:54.340140 16451 raft_consensus.cc:2749] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.344640 16298 raft_consensus.cc:2468] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:54.354854 16416 tablet_bootstrap.cc:492] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:54.355468 16416 ts_tablet_manager.cc:1403] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.290s	user 0.013s	sys 0.012s
I20260709 13:06:54.358551 16417 tablet_bootstrap.cc:492] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:54.359084 16337 leader_election.cc:304] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:54.359936 16452 raft_consensus.cc:2804] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.359129 16417 ts_tablet_manager.cc:1403] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.001s
I20260709 13:06:54.358197 16416 raft_consensus.cc:359] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.361006 16416 raft_consensus.cc:385] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.361297 16416 raft_consensus.cc:740] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.361971 16416 consensus_queue.cc:260] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.363592 16418 ts_tablet_manager.cc:1434] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.000s
I20260709 13:06:54.368541 16418 tablet_bootstrap.cc:492] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:54.369833 16416 ts_tablet_manager.cc:1434] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 13:06:54.364423 16452 raft_consensus.cc:697] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:54.373584 16452 consensus_queue.cc:237] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.378367 16417 raft_consensus.cc:359] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.382458 16498 raft_consensus.cc:493] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.384006 16445 raft_consensus.cc:493] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.379287 16417 raft_consensus.cc:385] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.384725 16417 raft_consensus.cc:740] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.384582 16445 raft_consensus.cc:515] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.383692 16498 raft_consensus.cc:515] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.389389 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd94eae6c1241e5838930e00eb43c4c" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:54.390532 16498 leader_election.cc:290] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:54.392182 16416 tablet_bootstrap.cc:492] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:54.393128 16504 consensus_queue.cc:1048] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.396155 16417 consensus_queue.cc:260] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:06:54.396989 16264 leader_election.cc:343] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.398695 16417 ts_tablet_manager.cc:1434] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.001s
I20260709 13:06:54.399935 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7973b570c8554e048bd5a9d02fd13a46" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:54.400640 16149 raft_consensus.cc:2468] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:54.401895 16190 leader_election.cc:304] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:54.403658 16540 raft_consensus.cc:2804] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.404078 16540 raft_consensus.cc:493] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.404414 16540 raft_consensus.cc:3060] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.409931 16417 tablet_bootstrap.cc:492] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:54.394626 16504 consensus_queue.cc:1048] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:06:54.432154 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7973b570c8554e048bd5a9d02fd13a46" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:54.432997 16372 raft_consensus.cc:2468] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:54.434203 16540 raft_consensus.cc:515] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.440703 16451 raft_consensus.cc:493] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.440755 16370 raft_consensus.cc:3060] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.441267 16451 raft_consensus.cc:515] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.443413 16451 leader_election.cc:290] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:54.445549 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76774e4153c7483683fcec5f557bfaad" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:54.417636 16445 leader_election.cc:290] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:54.438932 16046 catalog_manager.cc:5697] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.454314 16415 tablet_bootstrap.cc:492] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:54.456060 16415 ts_tablet_manager.cc:1403] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.127s	user 0.021s	sys 0.021s
I20260709 13:06:54.451774 16418 tablet_bootstrap.cc:654] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.465679 16415 raft_consensus.cc:359] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.466820 16415 raft_consensus.cc:385] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.467198 16415 raft_consensus.cc:740] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.469674 16538 raft_consensus.cc:493] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.470211 16417 tablet_bootstrap.cc:654] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.470245 16538 raft_consensus.cc:515] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.476940 16538 leader_election.cc:290] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:54.478307 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:54.478525 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:54.479974 16188 leader_election.cc:343] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:54.480499 16190 leader_election.cc:343] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.480911 16190 leader_election.cc:304] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:06:54.482333 16538 raft_consensus.cc:2749] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.484431 16540 leader_election.cc:290] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:54.485639 16416 tablet_bootstrap.cc:654] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.474208 16415 consensus_queue.cc:260] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.488006 16604 raft_consensus.cc:493] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.488451 16619 raft_consensus.cc:493] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.488530 16604 raft_consensus.cc:515] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.493263 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7973b570c8554e048bd5a9d02fd13a46" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:54.493952 16373 raft_consensus.cc:3060] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.490191 16619 raft_consensus.cc:515] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.497972 16604 leader_election.cc:290] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:54.498772 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2201023e16254eed9712e5fba24caa38" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.500281 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2201023e16254eed9712e5fba24caa38" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:54.500419 16113 leader_election.cc:343] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.501044 16298 raft_consensus.cc:2468] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:54.501523 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7973b570c8554e048bd5a9d02fd13a46" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:54.502097 16148 raft_consensus.cc:3060] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.503371 16114 leader_election.cc:304] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:54.504913 16507 raft_consensus.cc:2804] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.505327 16507 raft_consensus.cc:493] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.505692 16507 raft_consensus.cc:3060] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.507480 16619 leader_election.cc:290] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:54.512012 16298 raft_consensus.cc:3060] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:54.458827 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 60444.9us
I20260709 13:06:54.525641 16415 ts_tablet_manager.cc:1434] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.001s
I20260709 13:06:54.533499 16500 raft_consensus.cc:493] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.534062 16500 raft_consensus.cc:515] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.538928 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "424dbaa8a798455b9ae4d2f0b0989561" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:54.548967 16415 tablet_bootstrap.cc:492] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:54.546650 16148 raft_consensus.cc:2468] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:54.556490 16190 leader_election.cc:304] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:54.558663 16500 leader_election.cc:290] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:54.559824 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c540d71b564c08b9032559d7975cc5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.543736 16507 raft_consensus.cc:515] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:06:54.561774 16336 leader_election.cc:343] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.561973 16415 tablet_bootstrap.cc:654] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.561563 16604 consensus_queue.cc:1048] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.540827 16373 raft_consensus.cc:2468] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:54.564827 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2201023e16254eed9712e5fba24caa38" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:54.556720 16418 tablet_bootstrap.cc:492] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
W20260709 13:06:54.566082 16113 leader_election.cc:343] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.566494 16713 raft_consensus.cc:2804] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.566488 16418 ts_tablet_manager.cc:1403] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.198s	user 0.021s	sys 0.000s
I20260709 13:06:54.567239 16713 raft_consensus.cc:697] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:54.572055 16507 leader_election.cc:290] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:54.574688 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2201023e16254eed9712e5fba24caa38" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:54.575533 16297 raft_consensus.cc:3060] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.597301 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd94eae6c1241e5838930e00eb43c4c" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.598034 16223 raft_consensus.cc:2468] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:54.599059 16417 tablet_bootstrap.cc:492] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:54.599521 16417 ts_tablet_manager.cc:1403] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.200s	user 0.008s	sys 0.010s
I20260709 13:06:54.604808 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76774e4153c7483683fcec5f557bfaad" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.611125 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c540d71b564c08b9032559d7975cc5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:54.613142 16262 leader_election.cc:304] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:54.613454 16418 raft_consensus.cc:359] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.614104 16418 raft_consensus.cc:385] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.614358 16418 raft_consensus.cc:740] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.614432 16262 leader_election.cc:304] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:54.615023 16418 consensus_queue.cc:260] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.609160 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "424dbaa8a798455b9ae4d2f0b0989561" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.615809 16451 raft_consensus.cc:2804] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.616278 16451 raft_consensus.cc:493] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.616611 16451 raft_consensus.cc:3060] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.631565 16418 ts_tablet_manager.cc:1434] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.000s
I20260709 13:06:54.619699 16417 raft_consensus.cc:359] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.633530 16417 raft_consensus.cc:385] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.633646 16416 tablet_bootstrap.cc:492] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:54.634124 16416 ts_tablet_manager.cc:1403] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.242s	user 0.005s	sys 0.018s
I20260709 13:06:54.633827 16417 raft_consensus.cc:740] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.620026 16710 raft_consensus.cc:2749] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.620187 16713 consensus_queue.cc:237] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
W20260709 13:06:54.620332 16262 leader_election.cc:343] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:54.630190 16338 leader_election.cc:343] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.628463 16297 raft_consensus.cc:2468] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:54.643731 16114 leader_election.cc:304] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:54.642545 16338 leader_election.cc:304] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:54.639010 16416 raft_consensus.cc:359] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.644870 16589 raft_consensus.cc:2804] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.646797 16589 raft_consensus.cc:697] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:54.644825 16451 raft_consensus.cc:515] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.647670 16589 consensus_queue.cc:237] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.649897 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd94eae6c1241e5838930e00eb43c4c" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:54.650149 16714 raft_consensus.cc:2749] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.646293 16416 raft_consensus.cc:385] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.651396 16416 raft_consensus.cc:740] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
W20260709 13:06:54.653878 16264 leader_election.cc:343] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:54.655123 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:06:54.646709 16262 leader_election.cc:343] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.662242 16262 leader_election.cc:304] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:54.655882 16452 raft_consensus.cc:493] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.664055 16604 raft_consensus.cc:2749] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.664024 16452 raft_consensus.cc:515] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.665040 16604 consensus_queue.cc:1048] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:54.666451 16452 leader_election.cc:290] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:54.650986 16417 consensus_queue.cc:260] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.669986 16416 consensus_queue.cc:260] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.650187 16451 leader_election.cc:290] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:54.675642 16418 tablet_bootstrap.cc:492] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:54.676357 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:54.678026 16416 ts_tablet_manager.cc:1434] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.044s	user 0.004s	sys 0.004s
W20260709 13:06:54.678756 16337 leader_election.cc:343] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.677104 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:54.680898 16417 ts_tablet_manager.cc:1434] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.081s	user 0.010s	sys 0.000s
W20260709 13:06:54.682579 16338 leader_election.cc:343] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.682996 16415 tablet_bootstrap.cc:492] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:54.683535 16415 ts_tablet_manager.cc:1403] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.152s	user 0.028s	sys 0.010s
I20260709 13:06:54.683072 16338 leader_election.cc:304] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:54.685600 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd94eae6c1241e5838930e00eb43c4c" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:54.686187 16223 raft_consensus.cc:3060] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.687476 16415 raft_consensus.cc:359] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.688743 16415 raft_consensus.cc:385] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.689107 16415 raft_consensus.cc:740] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.689966 16415 consensus_queue.cc:260] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.697952 16417 tablet_bootstrap.cc:492] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:54.732203 16418 tablet_bootstrap.cc:654] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.705540 16045 catalog_manager.cc:5697] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:54.736060 16589 raft_consensus.cc:493] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.736598 16589 raft_consensus.cc:515] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.737102 16418 tablet_bootstrap.cc:492] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:54.737593 16418 ts_tablet_manager.cc:1403] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.091s	user 0.024s	sys 0.007s
I20260709 13:06:54.738734 16589 leader_election.cc:290] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:54.740574 16418 raft_consensus.cc:359] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.741278 16418 raft_consensus.cc:385] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.741556 16418 raft_consensus.cc:740] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.742172 16418 consensus_queue.cc:260] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.742601 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:54.742074 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:54.744513 16113 leader_election.cc:343] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:54.745990 16114 leader_election.cc:343] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.746424 16114 leader_election.cc:304] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:54.747577 16507 raft_consensus.cc:2749] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.754900 16418 ts_tablet_manager.cc:1434] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 13:06:54.698483 16416 tablet_bootstrap.cc:492] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:54.760027 16418 tablet_bootstrap.cc:492] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:54.729789 16223 raft_consensus.cc:2468] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:54.731350 16297 raft_consensus.cc:3060] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.701457 16719 raft_consensus.cc:2749] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.707280 16415 ts_tablet_manager.cc:1434] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:06:54.718261 16504 raft_consensus.cc:493] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ce6da6d71124256be0679451014f776)
I20260709 13:06:54.769877 16418 tablet_bootstrap.cc:654] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.770203 16415 tablet_bootstrap.cc:492] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:54.772094 16589 raft_consensus.cc:493] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.772578 16589 raft_consensus.cc:515] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.774608 16589 leader_election.cc:290] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:54.775045 16504 raft_consensus.cc:515] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.779846 16418 tablet_bootstrap.cc:492] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:54.780776 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c540d71b564c08b9032559d7975cc5" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:54.776568 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c540d71b564c08b9032559d7975cc5" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.781471 16372 raft_consensus.cc:2468] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
W20260709 13:06:54.782857 16113 leader_election.cc:343] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.783311 16451 raft_consensus.cc:493] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.783527 16113 leader_election.cc:304] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:54.784293 16714 raft_consensus.cc:493] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.784677 16589 raft_consensus.cc:2804] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.785106 16589 raft_consensus.cc:493] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.784806 16714 raft_consensus.cc:515] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.785486 16589 raft_consensus.cc:3060] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.787164 16714 leader_election.cc:290] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:54.789191 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.789929 16224 raft_consensus.cc:2468] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:54.790943 16415 tablet_bootstrap.cc:654] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.791219 16336 leader_election.cc:304] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:54.791925 16451 raft_consensus.cc:515] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.792615 16589 raft_consensus.cc:515] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.793915 16451 leader_election.cc:290] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:54.795094 16504 leader_election.cc:290] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:54.795727 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b521a909fd464b72bc2742ff3157658b" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:54.798000 16719 raft_consensus.cc:2804] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.798408 16719 raft_consensus.cc:493] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.798748 16719 raft_consensus.cc:3060] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.798858 16262 leader_election.cc:304] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:54.800522 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c540d71b564c08b9032559d7975cc5" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:54.800787 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b521a909fd464b72bc2742ff3157658b" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.801061 16710 raft_consensus.cc:2804] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.801764 16710 raft_consensus.cc:697] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:54.802546 16336 leader_election.cc:304] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:54.802719 16710 consensus_queue.cc:237] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.801123 16372 raft_consensus.cc:3060] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.780407 16418 ts_tablet_manager.cc:1403] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.006s
I20260709 13:06:54.806844 16452 raft_consensus.cc:2749] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:06:54.724449 16047 catalog_manager.cc:5697] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.809397 16589 leader_election.cc:290] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:54.815052 16417 tablet_bootstrap.cc:654] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.815459 16416 tablet_bootstrap.cc:654] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.818390 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c540d71b564c08b9032559d7975cc5" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
W20260709 13:06:54.842941 16113 leader_election.cc:343] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.787672 16714 consensus_queue.cc:1048] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:06:54.819892 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:54.820343 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e24ed51cb74d0da9f8b77dcbdab5b9" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:54.845885 16370 raft_consensus.cc:2468] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:54.821039 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e24ed51cb74d0da9f8b77dcbdab5b9" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:54.852805 16262 leader_election.cc:304] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:54.852150 16498 raft_consensus.cc:493] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.853085 16418 raft_consensus.cc:359] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.854111 16418 raft_consensus.cc:385] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.854022 16498 raft_consensus.cc:515] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.854542 16418 raft_consensus.cc:740] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.856151 16498 leader_election.cc:290] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
W20260709 13:06:54.856267 16264 leader_election.cc:343] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.856834 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cec4e308d104a9f9049fa0c2263b58d" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:54.857843 16338 leader_election.cc:343] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.858109 16298 raft_consensus.cc:2393] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce6da6d71124256be0679451014f776 in current term 1: Already voted for candidate c8106927ed954e1c800869427b6c7964 in this term.
I20260709 13:06:54.859463 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cec4e308d104a9f9049fa0c2263b58d" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:54.861405 16451 raft_consensus.cc:2804] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:54.861812 16451 raft_consensus.cc:493] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:54.862125 16451 raft_consensus.cc:3060] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.862975 16188 leader_election.cc:304] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:54.864619 16713 raft_consensus.cc:3060] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.869400 16418 consensus_queue.cc:260] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.873651 16415 tablet_bootstrap.cc:492] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:54.874382 16415 ts_tablet_manager.cc:1403] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.104s	user 0.009s	sys 0.008s
I20260709 13:06:54.876250 16713 raft_consensus.cc:2749] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:54.876873 16451 raft_consensus.cc:515] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.877586 16415 raft_consensus.cc:359] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.878255 16415 raft_consensus.cc:385] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.878512 16415 raft_consensus.cc:740] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.879144 16415 consensus_queue.cc:260] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:54.881558 16415 ts_tablet_manager.cc:1434] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:06:54.882741 16415 tablet_bootstrap.cc:492] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:54.827698 16372 raft_consensus.cc:2468] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:54.886401 16418 ts_tablet_manager.cc:1434] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.001s
I20260709 13:06:54.888386 16113 leader_election.cc:304] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:54.890021 16731 raft_consensus.cc:2804] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.895674 16451 leader_election.cc:290] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:54.896641 16418 tablet_bootstrap.cc:492] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:54.892514 16731 raft_consensus.cc:697] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:54.898795 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e24ed51cb74d0da9f8b77dcbdab5b9" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:54.874342 16719 raft_consensus.cc:515] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.903178 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e24ed51cb74d0da9f8b77dcbdab5b9" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:54.903842 16372 raft_consensus.cc:3060] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:54.902776 16264 leader_election.cc:343] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.904908 16452 consensus_queue.cc:1048] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:54.913144 16372 raft_consensus.cc:2468] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:54.914106 16417 tablet_bootstrap.cc:492] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:54.914649 16417 ts_tablet_manager.cc:1403] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.217s	user 0.010s	sys 0.004s
I20260709 13:06:54.911659 16731 consensus_queue.cc:237] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.920540 16262 leader_election.cc:304] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:54.921586 16418 tablet_bootstrap.cc:654] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.920565 16416 tablet_bootstrap.cc:492] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:54.922399 16416 ts_tablet_manager.cc:1403] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.226s	user 0.009s	sys 0.011s
I20260709 13:06:54.924556 16710 raft_consensus.cc:2804] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.925060 16710 raft_consensus.cc:697] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:54.924854 16417 raft_consensus.cc:359] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.925576 16417 raft_consensus.cc:385] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.925859 16417 raft_consensus.cc:740] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.925889 16710 consensus_queue.cc:237] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:54.925729 16719 leader_election.cc:290] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:54.928422 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:54.928665 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:54.929320 16223 raft_consensus.cc:3060] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:54.926635 16417 consensus_queue.cc:260] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.937534 16415 tablet_bootstrap.cc:654] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:54.948096 16338 leader_election.cc:343] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.939700 16048 catalog_manager.cc:5697] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:54.951591 16416 raft_consensus.cc:359] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.952353 16416 raft_consensus.cc:385] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:54.952629 16416 raft_consensus.cc:740] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:54.953703 16417 ts_tablet_manager.cc:1434] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.004s
I20260709 13:06:54.955272 16416 consensus_queue.cc:260] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:54.959161 16416 ts_tablet_manager.cc:1434] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:06:54.963733 16416 tablet_bootstrap.cc:492] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:54.966639 16500 raft_consensus.cc:493] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:54.967165 16500 raft_consensus.cc:515] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:54.968847 16417 tablet_bootstrap.cc:492] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:54.969359 16500 leader_election.cc:290] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:54.972433 16223 raft_consensus.cc:2468] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:54.974058 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "424dbaa8a798455b9ae4d2f0b0989561" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:54.974891 16297 raft_consensus.cc:2468] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:54.975395 16417 tablet_bootstrap.cc:654] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:54.978425 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "424dbaa8a798455b9ae4d2f0b0989561" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:54.978534 16336 leader_election.cc:304] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:54.972481 16416 tablet_bootstrap.cc:654] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:54.996461 16336 leader_election.cc:343] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:54.993500 16500 raft_consensus.cc:2804] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:54.999662 16500 raft_consensus.cc:697] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:55.000453 16500 consensus_queue.cc:237] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.010368 16048 catalog_manager.cc:5697] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.017696 16337 leader_election.cc:304] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:55.019382 16452 raft_consensus.cc:2804] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.019870 16452 raft_consensus.cc:493] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.020227 16452 raft_consensus.cc:3060] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.019444 16045 catalog_manager.cc:5697] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.020709 16418 tablet_bootstrap.cc:492] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:55.021217 16418 ts_tablet_manager.cc:1403] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.129s	user 0.010s	sys 0.009s
I20260709 13:06:55.032578 16418 raft_consensus.cc:359] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.033340 16418 raft_consensus.cc:385] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.033631 16418 raft_consensus.cc:740] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.034629 16418 consensus_queue.cc:260] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.036993 16418 ts_tablet_manager.cc:1434] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:06:55.049418 16418 tablet_bootstrap.cc:492] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:55.052556 16415 tablet_bootstrap.cc:492] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:55.053321 16415 ts_tablet_manager.cc:1403] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.171s	user 0.014s	sys 0.011s
I20260709 13:06:55.061163 16452 raft_consensus.cc:515] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.063184 16452 leader_election.cc:290] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:55.064075 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "424dbaa8a798455b9ae4d2f0b0989561" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
W20260709 13:06:55.066262 16336 leader_election.cc:343] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.067196 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "424dbaa8a798455b9ae4d2f0b0989561" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:55.067935 16297 raft_consensus.cc:3060] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.076241 16713 raft_consensus.cc:493] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.076824 16713 raft_consensus.cc:515] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.081349 16713 leader_election.cc:290] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:55.082422 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047f65a57d24cafbf3f41f5b235556e" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:55.084571 16297 raft_consensus.cc:2468] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
W20260709 13:06:55.084718 16188 leader_election.cc:343] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.086354 16417 tablet_bootstrap.cc:492] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:55.086869 16417 ts_tablet_manager.cc:1403] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.126s	user 0.008s	sys 0.020s
I20260709 13:06:55.085990 16337 leader_election.cc:304] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776
I20260709 13:06:55.089108 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047f65a57d24cafbf3f41f5b235556e" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:55.090912 16190 leader_election.cc:343] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.091346 16190 leader_election.cc:304] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:06:55.092489 16713 raft_consensus.cc:2749] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.093894 16418 tablet_bootstrap.cc:654] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.097486 16417 raft_consensus.cc:359] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.098212 16417 raft_consensus.cc:385] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.098474 16417 raft_consensus.cc:740] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.099119 16417 consensus_queue.cc:260] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.091347 16415 raft_consensus.cc:359] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.101300 16415 raft_consensus.cc:385] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.101691 16415 raft_consensus.cc:740] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.100659 16452 raft_consensus.cc:2804] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.103658 16452 raft_consensus.cc:697] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:55.104756 16452 consensus_queue.cc:237] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.102759 16415 consensus_queue.cc:260] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.116782 16417 ts_tablet_manager.cc:1434] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:06:55.117944 16417 tablet_bootstrap.cc:492] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:55.123718 16417 tablet_bootstrap.cc:654] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.114279 16045 catalog_manager.cc:5697] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:55.131268 16415 ts_tablet_manager.cc:1434] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.078s	user 0.005s	sys 0.004s
I20260709 13:06:55.140971 16415 tablet_bootstrap.cc:492] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:55.147149 16731 raft_consensus.cc:493] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.147715 16731 raft_consensus.cc:515] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.150482 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:55.152644 16731 leader_election.cc:290] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:55.154198 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:55.155061 16224 raft_consensus.cc:2393] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa8eb07867e7420b80a7e615d570509d in current term 1: Already voted for candidate c8106927ed954e1c800869427b6c7964 in this term.
I20260709 13:06:55.156440 16113 leader_election.cc:304] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:55.157847 16731 raft_consensus.cc:3060] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.164525 16713 consensus_queue.cc:1048] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.165695 16589 raft_consensus.cc:493] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.166303 16589 raft_consensus.cc:515] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.169778 16589 leader_election.cc:290] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:55.169804 16415 tablet_bootstrap.cc:654] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.171283 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c52cafca664c5f90589901f62e4b16" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:55.172610 16731 raft_consensus.cc:2749] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.172863 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c52cafca664c5f90589901f62e4b16" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:55.170498 16416 tablet_bootstrap.cc:492] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:55.173739 16416 ts_tablet_manager.cc:1403] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.210s	user 0.036s	sys 0.033s
W20260709 13:06:55.173926 16114 leader_election.cc:343] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:55.174471 16113 leader_election.cc:343] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.175067 16113 leader_election.cc:304] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:55.176246 16731 raft_consensus.cc:2749] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.176640 16416 raft_consensus.cc:359] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.177312 16416 raft_consensus.cc:385] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.182723 16418 tablet_bootstrap.cc:492] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:55.183549 16418 ts_tablet_manager.cc:1403] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.141s	user 0.016s	sys 0.008s
I20260709 13:06:55.177582 16416 raft_consensus.cc:740] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.187849 16416 consensus_queue.cc:260] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.193316 16416 ts_tablet_manager.cc:1434] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:06:55.215813 16498 consensus_queue.cc:1048] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.234069 16418 raft_consensus.cc:359] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.234754 16418 raft_consensus.cc:385] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.234993 16418 raft_consensus.cc:740] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.237481 16416 tablet_bootstrap.cc:492] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:55.241693 16048 catalog_manager.cc:5697] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } }
W20260709 13:06:55.246151 16113 consensus_peers.cc:597] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:55.251469 16416 tablet_bootstrap.cc:654] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.253955 16731 consensus_queue.cc:1048] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.235558 16418 consensus_queue.cc:260] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.272189 16418 ts_tablet_manager.cc:1434] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.000s
I20260709 13:06:55.272324 16415 tablet_bootstrap.cc:492] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:55.272996 16415 ts_tablet_manager.cc:1403] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.136s	user 0.033s	sys 0.017s
I20260709 13:06:55.273355 16418 tablet_bootstrap.cc:492] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:55.276188 16415 raft_consensus.cc:359] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.277002 16415 raft_consensus.cc:385] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.277276 16415 raft_consensus.cc:740] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.278259 16415 consensus_queue.cc:260] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.285022 16415 ts_tablet_manager.cc:1434] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:06:55.285166 16416 tablet_bootstrap.cc:492] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:55.285679 16416 ts_tablet_manager.cc:1403] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.001s
I20260709 13:06:55.286463 16418 tablet_bootstrap.cc:654] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.288373 16416 raft_consensus.cc:359] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.294620 16415 tablet_bootstrap.cc:492] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:55.294898 16418 tablet_bootstrap.cc:492] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:55.291604 16416 raft_consensus.cc:385] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.295567 16418 ts_tablet_manager.cc:1403] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.003s
I20260709 13:06:55.295534 16416 raft_consensus.cc:740] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.296459 16416 consensus_queue.cc:260] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.298623 16418 raft_consensus.cc:359] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.299630 16418 raft_consensus.cc:385] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.299912 16418 raft_consensus.cc:740] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.301026 16731 raft_consensus.cc:493] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.301527 16731 raft_consensus.cc:515] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.304009 16418 consensus_queue.cc:260] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.303574 16731 leader_election.cc:290] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:55.305718 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd94eae6c1241e5838930e00eb43c4c" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:55.306504 16418 ts_tablet_manager.cc:1434] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:06:55.305272 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dd94eae6c1241e5838930e00eb43c4c" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:55.307919 16415 tablet_bootstrap.cc:654] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.308209 16224 raft_consensus.cc:2393] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa8eb07867e7420b80a7e615d570509d in current term 1: Already voted for candidate 115c2d35eda349178144f8365481e89c in this term.
I20260709 13:06:55.308257 16417 tablet_bootstrap.cc:492] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:55.308878 16417 ts_tablet_manager.cc:1403] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.191s	user 0.030s	sys 0.026s
I20260709 13:06:55.309432 16113 leader_election.cc:304] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:55.311142 16589 raft_consensus.cc:3060] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.314031 16417 raft_consensus.cc:359] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.314721 16417 raft_consensus.cc:385] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.314985 16417 raft_consensus.cc:740] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.317067 16416 ts_tablet_manager.cc:1434] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 13:06:55.318109 16416 tablet_bootstrap.cc:492] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:55.326136 16710 consensus_queue.cc:1048] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.325405 16417 consensus_queue.cc:260] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.330668 16445 raft_consensus.cc:493] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.331148 16445 raft_consensus.cc:515] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.333235 16445 leader_election.cc:290] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:55.334003 16417 ts_tablet_manager.cc:1434] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:06:55.334262 16452 raft_consensus.cc:493] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.334695 16452 raft_consensus.cc:515] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.335057 16417 tablet_bootstrap.cc:492] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:55.336618 16452 leader_election.cc:290] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:55.338399 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:55.340179 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b7aa0c7f8b4bf4a2e4881b23351045" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:55.340857 16150 raft_consensus.cc:2468] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:55.341835 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:55.343501 16337 leader_election.cc:343] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.344754 16264 leader_election.cc:304] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:55.346307 16604 raft_consensus.cc:2804] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.346727 16604 raft_consensus.cc:493] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.347191 16604 raft_consensus.cc:3060] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.353063 16589 raft_consensus.cc:2749] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.359844 16451 consensus_queue.cc:1048] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.364038 16710 consensus_queue.cc:1048] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.323951 16416 tablet_bootstrap.cc:654] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.307621 16418 tablet_bootstrap.cc:492] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:55.392235 16498 raft_consensus.cc:493] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.392933 16498 raft_consensus.cc:515] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.397878 16417 tablet_bootstrap.cc:654] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.400269 16604 raft_consensus.cc:515] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.402428 16604 leader_election.cc:290] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:55.394757 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b7aa0c7f8b4bf4a2e4881b23351045" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:55.403992 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b7aa0c7f8b4bf4a2e4881b23351045" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:55.404654 16148 raft_consensus.cc:3060] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.404556 16225 raft_consensus.cc:2468] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:55.410903 16452 raft_consensus.cc:493] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.411593 16452 raft_consensus.cc:515] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.462724 16148 raft_consensus.cc:2468] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:55.482251 16498 leader_election.cc:290] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:55.482669 16416 tablet_bootstrap.cc:492] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:55.483223 16416 ts_tablet_manager.cc:1403] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.165s	user 0.014s	sys 0.007s
I20260709 13:06:55.485910 16416 raft_consensus.cc:359] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.486548 16416 raft_consensus.cc:385] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.486804 16416 raft_consensus.cc:740] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.487396 16416 consensus_queue.cc:260] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.487720 16452 leader_election.cc:290] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:55.489578 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047f65a57d24cafbf3f41f5b235556e" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:55.490984 16338 leader_election.cc:343] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.491819 16416 ts_tablet_manager.cc:1434] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:06:55.528582 16264 leader_election.cc:304] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:55.529893 16710 raft_consensus.cc:2804] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:55.538345 16416 tablet_bootstrap.cc:492] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
W20260709 13:06:55.514073 16336 leader_election.cc:343] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.521047 16604 raft_consensus.cc:493] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.555843 16604 raft_consensus.cc:515] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.571460 16604 leader_election.cc:290] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:55.573494 16418 tablet_bootstrap.cc:654] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.548846 16710 raft_consensus.cc:697] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:55.587281 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "424dbaa8a798455b9ae4d2f0b0989561" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:55.590279 16416 tablet_bootstrap.cc:654] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.548719 16336 leader_election.cc:304] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:55.602808 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "424dbaa8a798455b9ae4d2f0b0989561" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:55.603683 16297 raft_consensus.cc:2393] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3ce6da6d71124256be0679451014f776 in current term 1: Already voted for candidate c8106927ed954e1c800869427b6c7964 in this term.
I20260709 13:06:55.604219 16224 raft_consensus.cc:3060] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.604625 16710 consensus_queue.cc:237] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.606653 16417 tablet_bootstrap.cc:492] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:55.607182 16417 ts_tablet_manager.cc:1403] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.272s	user 0.026s	sys 0.036s
I20260709 13:06:55.611204 16604 raft_consensus.cc:493] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.611785 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:55.611716 16604 raft_consensus.cc:515] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.612468 16371 raft_consensus.cc:2468] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:55.617728 16604 leader_election.cc:290] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:55.618091 16714 raft_consensus.cc:2749] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.622367 16507 consensus_queue.cc:1048] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.623543 16416 tablet_bootstrap.cc:492] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:55.623721 16507 consensus_queue.cc:1048] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.624053 16416 ts_tablet_manager.cc:1403] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.109s	user 0.013s	sys 0.015s
I20260709 13:06:55.624167 16417 raft_consensus.cc:359] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.625130 16417 raft_consensus.cc:385] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.625447 16417 raft_consensus.cc:740] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.626288 16417 consensus_queue.cc:260] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.626726 16416 raft_consensus.cc:359] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.627399 16416 raft_consensus.cc:385] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.627682 16416 raft_consensus.cc:740] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.628470 16416 consensus_queue.cc:260] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:55.630112 16415 tablet_bootstrap.cc:492] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:55.630126 16416 ts_tablet_manager.cc:1434] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:06:55.630781 16415 ts_tablet_manager.cc:1403] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.336s	user 0.087s	sys 0.092s
I20260709 13:06:55.636121 16416 tablet_bootstrap.cc:492] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:55.641000 16417 ts_tablet_manager.cc:1434] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.000s
I20260709 13:06:55.651013 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:55.635396 16188 leader_election.cc:304] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:55.654129 16762 raft_consensus.cc:3060] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:55.645326 16336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 137816us
W20260709 13:06:55.669211 16264 consensus_peers.cc:597] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:55.680773 16417 tablet_bootstrap.cc:492] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:55.678170 16762 raft_consensus.cc:2749] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:55.697263 16415 raft_consensus.cc:359] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.697826 16415 raft_consensus.cc:385] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.698050 16415 raft_consensus.cc:740] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.690199 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:55.699519 16373 raft_consensus.cc:2468] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:55.696448 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047f65a57d24cafbf3f41f5b235556e" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:55.703241 16224 raft_consensus.cc:2468] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
W20260709 13:06:55.706552 16757 log.cc:927] Time spent T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964: Append to log took a long time: real 0.178s	user 0.003s	sys 0.000s
I20260709 13:06:55.690393 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:55.707376 16147 raft_consensus.cc:2468] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:55.708302 16415 consensus_queue.cc:260] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.707638 16418 tablet_bootstrap.cc:492] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:55.709388 16418 ts_tablet_manager.cc:1403] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.402s	user 0.019s	sys 0.009s
I20260709 13:06:55.710635 16415 ts_tablet_manager.cc:1434] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.080s	user 0.006s	sys 0.002s
I20260709 13:06:55.715562 16418 raft_consensus.cc:359] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.717140 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b7aa0c7f8b4bf4a2e4881b23351045" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:55.717798 16418 raft_consensus.cc:385] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.718902 16418 raft_consensus.cc:740] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.720480 16418 consensus_queue.cc:260] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.717847 16221 raft_consensus.cc:3060] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.716699 16415 tablet_bootstrap.cc:492] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:55.732117 16416 tablet_bootstrap.cc:654] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.754379 16417 tablet_bootstrap.cc:654] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:55.694769 16264 leader_election.cc:343] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.782976 16418 ts_tablet_manager.cc:1434] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.000s
W20260709 13:06:55.780239 16770 log.cc:927] Time spent T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c: Append to log took a long time: real 0.078s	user 0.006s	sys 0.008s
W20260709 13:06:55.783841 16264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 90711.6us
I20260709 13:06:55.792160 16418 tablet_bootstrap.cc:492] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
W20260709 13:06:55.779434 16337 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 97311.5us
I20260709 13:06:55.795068 16264 leader_election.cc:304] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:55.805063 16731 raft_consensus.cc:493] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.805662 16771 raft_consensus.cc:2804] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.816084 16416 tablet_bootstrap.cc:492] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:55.816594 16416 ts_tablet_manager.cc:1403] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.181s	user 0.030s	sys 0.007s
I20260709 13:06:55.813639 16771 raft_consensus.cc:493] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.825758 16221 raft_consensus.cc:2468] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:55.834553 16776 consensus_queue.cc:1048] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.836632 16416 raft_consensus.cc:359] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.837332 16416 raft_consensus.cc:385] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.837615 16416 raft_consensus.cc:740] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.839131 16416 consensus_queue.cc:260] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.814352 16731 raft_consensus.cc:515] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.839886 16262 leader_election.cc:304] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
W20260709 13:06:55.840770 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 146318us
I20260709 13:06:55.841439 16731 leader_election.cc:290] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:55.841573 16416 ts_tablet_manager.cc:1434] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:06:55.842615 16416 tablet_bootstrap.cc:492] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:55.842829 16710 raft_consensus.cc:2804] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.843247 16710 raft_consensus.cc:493] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.843605 16710 raft_consensus.cc:3060] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.844717 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:55.843256 16417 tablet_bootstrap.cc:492] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:55.846153 16417 ts_tablet_manager.cc:1403] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.196s	user 0.017s	sys 0.001s
I20260709 13:06:55.848865 16416 tablet_bootstrap.cc:654] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.855355 16498 raft_consensus.cc:493] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.856086 16498 raft_consensus.cc:515] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:55.857050 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:55.857194 16500 consensus_queue.cc:1048] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.857735 16371 raft_consensus.cc:2468] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:55.858994 16113 leader_election.cc:304] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:55.858741 16498 leader_election.cc:290] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:55.860671 16764 raft_consensus.cc:2804] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.861053 16764 raft_consensus.cc:493] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.861419 16764 raft_consensus.cc:3060] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.865644 16772 consensus_queue.cc:1048] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:55.867558 16336 leader_election.cc:304] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:55.870227 16775 raft_consensus.cc:2804] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:55.870641 16775 raft_consensus.cc:493] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:55.870994 16775 raft_consensus.cc:3060] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.824244 16771 raft_consensus.cc:3060] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.882524 16500 consensus_queue.cc:1048] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:55.885073 16451 raft_consensus.cc:493] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:55.906232 16451 raft_consensus.cc:515] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.908594 16451 leader_election.cc:290] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:55.875735 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:55.889516 16219 raft_consensus.cc:3060] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.892513 16417 raft_consensus.cc:359] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.912295 16417 raft_consensus.cc:385] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.912634 16417 raft_consensus.cc:740] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.913383 16417 consensus_queue.cc:260] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.916915 16415 tablet_bootstrap.cc:654] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.922164 16416 tablet_bootstrap.cc:492] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:55.922819 16416 ts_tablet_manager.cc:1403] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.080s	user 0.031s	sys 0.014s
I20260709 13:06:55.887506 16418 tablet_bootstrap.cc:654] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:55.910490 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bbd6b3bdff467f9d48329073cf0167" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:55.926962 16264 leader_election.cc:343] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.928236 16764 raft_consensus.cc:515] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.926519 16416 raft_consensus.cc:359] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.934767 16416 raft_consensus.cc:385] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:55.935253 16416 raft_consensus.cc:740] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:55.936091 16416 consensus_queue.cc:260] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.937582 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:55.938480 16416 ts_tablet_manager.cc:1434] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:06:55.940017 16764 leader_election.cc:290] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:55.940119 16416 tablet_bootstrap.cc:492] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:55.941210 16371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:55.944284 16371 raft_consensus.cc:3060] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:55.927765 16710 raft_consensus.cc:515] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:55.946972 16417 ts_tablet_manager.cc:1434] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.101s	user 0.004s	sys 0.004s
I20260709 13:06:55.952431 16295 raft_consensus.cc:2393] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa8eb07867e7420b80a7e615d570509d in current term 1: Already voted for candidate 115c2d35eda349178144f8365481e89c in this term.
W20260709 13:06:55.965422 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 123555us
I20260709 13:06:55.968055 16417 tablet_bootstrap.cc:492] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
W20260709 13:06:55.966796 16189 leader_election.cc:343] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:55.931186 16775 raft_consensus.cc:515] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:55.993239 16775 leader_election.cc:290] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:55.994616 16771 raft_consensus.cc:515] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:56.028002 16417 tablet_bootstrap.cc:654] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
W20260709 13:06:56.028426 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:56.039495 16416 tablet_bootstrap.cc:654] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.049750 16045 catalog_manager.cc:5697] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:56.050961 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047f65a57d24cafbf3f41f5b235556e" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:56.048102 16710 leader_election.cc:290] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:56.038132 16368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:56.060755 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:56.086259 16369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bbd6b3bdff467f9d48329073cf0167" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:56.087076 16369 raft_consensus.cc:2468] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
W20260709 13:06:56.089134 16783 log.cc:927] Time spent T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:06:56.070930 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:56.094368 16370 raft_consensus.cc:3060] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:56.096781 16784 log.cc:927] Time spent T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Append to log took a long time: real 0.080s	user 0.003s	sys 0.000s
I20260709 13:06:56.107251 16262 leader_election.cc:304] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:56.110687 16370 raft_consensus.cc:2468] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:56.111649 16451 raft_consensus.cc:2804] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.112072 16451 raft_consensus.cc:493] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.112370 16451 raft_consensus.cc:3060] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.128415 16418 tablet_bootstrap.cc:492] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:56.128937 16415 tablet_bootstrap.cc:492] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:56.129225 16418 ts_tablet_manager.cc:1403] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.340s	user 0.027s	sys 0.015s
I20260709 13:06:56.129666 16415 ts_tablet_manager.cc:1403] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.418s	user 0.021s	sys 0.030s
I20260709 13:06:56.133193 16418 raft_consensus.cc:359] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:56.134210 16418 raft_consensus.cc:385] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.134581 16418 raft_consensus.cc:740] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.085312 16371 raft_consensus.cc:2468] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:56.113379 16150 raft_consensus.cc:2393] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 115c2d35eda349178144f8365481e89c in current term 1: Already voted for candidate aa8eb07867e7420b80a7e615d570509d in this term.
I20260709 13:06:56.135463 16418 consensus_queue.cc:260] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:56.100557 16766 consensus_queue.cc:1048] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 13:06:56.168655 16113 leader_election.cc:304] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:56.170162 16619 raft_consensus.cc:2804] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.170683 16619 raft_consensus.cc:697] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:56.171496 16619 consensus_queue.cc:237] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:56.177134 16364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:56.177922 16364 raft_consensus.cc:2468] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:56.179047 16264 leader_election.cc:304] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:06:56.157346 16415 raft_consensus.cc:359] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.182337 16415 raft_consensus.cc:385] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.182637 16415 raft_consensus.cc:740] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.185164 16445 raft_consensus.cc:2749] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.188038 16416 tablet_bootstrap.cc:492] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:56.187875 16415 consensus_queue.cc:260] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.188644 16416 ts_tablet_manager.cc:1403] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.249s	user 0.015s	sys 0.003s
W20260709 13:06:56.189893 16338 leader_election.cc:343] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.193090 16415 ts_tablet_manager.cc:1434] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.000s
I20260709 13:06:56.198226 16418 ts_tablet_manager.cc:1434] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.005s
I20260709 13:06:56.198710 16415 tablet_bootstrap.cc:492] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
W20260709 13:06:56.204298 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 90835.6us
I20260709 13:06:56.206243 16451 raft_consensus.cc:515] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:56.205297 16262 leader_election.cc:304] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:56.210419 16418 tablet_bootstrap.cc:492] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:56.217617 16451 leader_election.cc:290] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:56.224918 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bbd6b3bdff467f9d48329073cf0167" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:56.207952 16710 raft_consensus.cc:2804] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.225661 16373 raft_consensus.cc:3060] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.246804 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047f65a57d24cafbf3f41f5b235556e" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:56.247553 16222 raft_consensus.cc:3060] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.248101 16373 raft_consensus.cc:2468] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:56.252676 16262 leader_election.cc:304] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:56.249439 16710 raft_consensus.cc:697] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:56.243430 16188 leader_election.cc:304] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:56.296572 16604 raft_consensus.cc:2804] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:56.304527 16338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 52347.7us
W20260709 13:06:56.326377 16188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 83071.3us
I20260709 13:06:56.327488 16809 raft_consensus.cc:2804] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.328153 16809 raft_consensus.cc:493] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.328624 16809 raft_consensus.cc:3060] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.337802 16416 raft_consensus.cc:359] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:56.338608 16416 raft_consensus.cc:385] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.338886 16416 raft_consensus.cc:740] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.309865 16604 raft_consensus.cc:697] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:56.339509 16416 consensus_queue.cc:260] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:56.343688 16416 ts_tablet_manager.cc:1434] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.155s	user 0.007s	sys 0.000s
I20260709 13:06:56.344759 16416 tablet_bootstrap.cc:492] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:56.352988 16415 tablet_bootstrap.cc:654] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.357932 16760 raft_consensus.cc:493] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.361938 16417 tablet_bootstrap.cc:492] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:56.362612 16417 ts_tablet_manager.cc:1403] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.404s	user 0.036s	sys 0.014s
I20260709 13:06:56.364630 16771 leader_election.cc:290] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:56.368062 16418 tablet_bootstrap.cc:654] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.368896 16417 raft_consensus.cc:359] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:56.369797 16417 raft_consensus.cc:385] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.370271 16417 raft_consensus.cc:740] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.374624 16619 raft_consensus.cc:493] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.375173 16619 raft_consensus.cc:515] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:56.376530 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:56.377199 16149 raft_consensus.cc:3060] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.377287 16619 leader_election.cc:290] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:56.381052 16604 consensus_queue.cc:237] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:56.371006 16417 consensus_queue.cc:260] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:56.376578 16222 raft_consensus.cc:2468] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:56.393035 16417 ts_tablet_manager.cc:1434] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.030s	user 0.001s	sys 0.005s
I20260709 13:06:56.394181 16417 tablet_bootstrap.cc:492] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:56.351557 16710 consensus_queue.cc:237] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:06:56.407917 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 74903us
I20260709 13:06:56.430771 16336 leader_election.cc:304] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:56.371412 16760 raft_consensus.cc:515] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:56.432967 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6959f2cf9714e91aee31bf53776915b" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:56.434721 16772 raft_consensus.cc:2804] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.435287 16772 raft_consensus.cc:697] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:56.436228 16772 consensus_queue.cc:237] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:56.429972 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bbd6b3bdff467f9d48329073cf0167" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:56.448962 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6959f2cf9714e91aee31bf53776915b" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:56.464572 16417 tablet_bootstrap.cc:654] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.487957 16498 raft_consensus.cc:493] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.504763 16418 tablet_bootstrap.cc:492] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:56.505366 16418 ts_tablet_manager.cc:1403] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.296s	user 0.022s	sys 0.028s
I20260709 13:06:56.488497 16498 raft_consensus.cc:515] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:56.530236 16769 raft_consensus.cc:493] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.530819 16769 raft_consensus.cc:515] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:56.532038 16809 raft_consensus.cc:515] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.532920 16769 leader_election.cc:290] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:56.533551 16760 leader_election.cc:290] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:06:56.552161 16336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 121510us
W20260709 13:06:56.559103 16113 leader_election.cc:343] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.556210 16809 leader_election.cc:290] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:56.564219 16418 raft_consensus.cc:359] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:56.565303 16418 raft_consensus.cc:385] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.565637 16418 raft_consensus.cc:740] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.565389 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:56.566141 16416 tablet_bootstrap.cc:654] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.566437 16418 consensus_queue.cc:260] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:56.567117 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:56.559832 16264 leader_election.cc:343] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:56.568720 16338 leader_election.cc:343] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.573513 16415 tablet_bootstrap.cc:492] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:56.573608 16498 leader_election.cc:290] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:56.574046 16415 ts_tablet_manager.cc:1403] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.376s	user 0.016s	sys 0.016s
I20260709 13:06:56.578795 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:56.580890 16366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:56.582827 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c52cafca664c5f90589901f62e4b16" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:56.583532 16147 raft_consensus.cc:2468] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
W20260709 13:06:56.584643 16337 leader_election.cc:343] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.585094 16337 leader_election.cc:304] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:56.586609 16769 raft_consensus.cc:2749] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.586745 16221 raft_consensus.cc:3060] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.588083 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:56.588721 16372 raft_consensus.cc:3060] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.590539 16190 leader_election.cc:304] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:06:56.598489 16190 leader_election.cc:304] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
W20260709 13:06:56.602226 16113 leader_election.cc:343] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.602898 16113 leader_election.cc:304] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:56.612811 16418 ts_tablet_manager.cc:1434] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.107s	user 0.007s	sys 0.003s
I20260709 13:06:56.612325 16044 catalog_manager.cc:5697] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:56.605489 16809 raft_consensus.cc:2804] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:56.613464 16619 raft_consensus.cc:2749] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.629737 16418 tablet_bootstrap.cc:492] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:56.640444 16797 consensus_queue.cc:1048] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:06:56.642058 16796 raft_consensus.cc:493] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.642618 16796 raft_consensus.cc:515] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.644712 16796 leader_election.cc:290] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:56.627128 16809 raft_consensus.cc:493] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:56.646912 16809 raft_consensus.cc:3060] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.653399 16415 raft_consensus.cc:359] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.655823 16415 raft_consensus.cc:385] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.656232 16415 raft_consensus.cc:740] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.661475 16417 tablet_bootstrap.cc:492] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:56.662063 16417 ts_tablet_manager.cc:1403] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.268s	user 0.005s	sys 0.021s
I20260709 13:06:56.658574 16415 consensus_queue.cc:260] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.676574 16149 raft_consensus.cc:2468] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:56.681229 16366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b15e00d79d42809ddd6a31dd9ee9f3" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
W20260709 13:06:56.684599 16262 leader_election.cc:343] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.685199 16415 ts_tablet_manager.cc:1434] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.111s	user 0.007s	sys 0.000s
I20260709 13:06:56.686146 16415 tablet_bootstrap.cc:492] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:56.705729 16710 raft_consensus.cc:493] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:56.706367 16710 raft_consensus.cc:515] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.708617 16710 leader_election.cc:290] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:56.712977 16507 consensus_queue.cc:1048] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:56.715368 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:56.716475 16366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7186244ab72e4b32ad068941b913f052" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:56.717381 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c52cafca664c5f90589901f62e4b16" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:56.718856 16372 raft_consensus.cc:2468] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:56.715122 16417 raft_consensus.cc:359] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:56.733451 16417 raft_consensus.cc:385] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.733773 16417 raft_consensus.cc:740] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.735035 16418 tablet_bootstrap.cc:654] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.744057 16417 consensus_queue.cc:260] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:56.746359 16417 ts_tablet_manager.cc:1434] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.001s
I20260709 13:06:56.747300 16417 tablet_bootstrap.cc:492] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:56.744061 16188 leader_election.cc:304] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:56.750603 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7186244ab72e4b32ad068941b913f052" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:56.751506 16808 raft_consensus.cc:2804] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:06:56.752913 16189 leader_election.cc:343] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:56.754930 16189 leader_election.cc:343] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.756103 16808 raft_consensus.cc:697] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:56.760504 16808 consensus_queue.cc:237] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.750509 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b15e00d79d42809ddd6a31dd9ee9f3" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:56.764864 16809 raft_consensus.cc:515] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:56.767835 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c52cafca664c5f90589901f62e4b16" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:56.768435 16148 raft_consensus.cc:3060] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:56.771718 16262 leader_election.cc:304] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:56.772704 16795 raft_consensus.cc:2749] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:06:56.773938 16262 leader_election.cc:343] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.774370 16262 leader_election.cc:304] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:56.774883 16416 tablet_bootstrap.cc:492] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:56.775465 16416 ts_tablet_manager.cc:1403] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.431s	user 0.031s	sys 0.037s
I20260709 13:06:56.776772 16710 raft_consensus.cc:2749] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.780849 16416 raft_consensus.cc:359] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.781625 16416 raft_consensus.cc:385] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.743497 16415 tablet_bootstrap.cc:654] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.783319 16416 raft_consensus.cc:740] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.782810 16148 raft_consensus.cc:2468] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:56.793464 16417 tablet_bootstrap.cc:654] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.795641 16416 consensus_queue.cc:260] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.802227 16190 leader_election.cc:304] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:56.804430 16047 catalog_manager.cc:5697] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:56.830488 16811 raft_consensus.cc:2804] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:56.783814 16809 leader_election.cc:290] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:56.840243 16047 catalog_manager.cc:5697] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:56.847085 16811 raft_consensus.cc:697] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:56.848078 16811 consensus_queue.cc:237] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:56.854647 16710 consensus_queue.cc:1048] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:56.861817 16416 ts_tablet_manager.cc:1434] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.086s	user 0.005s	sys 0.002s
I20260709 13:06:56.872390 16048 catalog_manager.cc:5697] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:56.874145 16416 tablet_bootstrap.cc:492] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:56.868546 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c52cafca664c5f90589901f62e4b16" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:56.912185 16416 tablet_bootstrap.cc:654] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:56.910653 16710 consensus_queue.cc:1048] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
W20260709 13:06:56.921262 16189 leader_election.cc:343] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:56.922469 16498 raft_consensus.cc:2749] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:56.923192 16415 tablet_bootstrap.cc:492] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:56.923697 16415 ts_tablet_manager.cc:1403] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.238s	user 0.007s	sys 0.003s
I20260709 13:06:56.926844 16415 raft_consensus.cc:359] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.929780 16415 raft_consensus.cc:385] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:56.930406 16415 raft_consensus.cc:740] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.931586 16415 consensus_queue.cc:260] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:56.934907 16415 ts_tablet_manager.cc:1434] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:06:56.942188 16764 consensus_queue.cc:1048] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:56.953984 16418 tablet_bootstrap.cc:492] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:56.954562 16418 ts_tablet_manager.cc:1403] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.331s	user 0.012s	sys 0.000s
I20260709 13:06:56.959434 16149 raft_consensus.cc:3060] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:56.959635 16264 consensus_peers.cc:597] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:56.962793 16044 catalog_manager.cc:5697] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
W20260709 13:06:57.029830 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 57868.7us
I20260709 13:06:57.029898 16415 tablet_bootstrap.cc:492] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:57.037721 16418 raft_consensus.cc:359] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:57.043850 16418 raft_consensus.cc:385] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.044286 16418 raft_consensus.cc:740] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:56.971175 16769 raft_consensus.cc:493] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.046838 16769 raft_consensus.cc:515] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:57.047919 16418 consensus_queue.cc:260] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:57.051328 16417 tablet_bootstrap.cc:492] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:57.051899 16417 ts_tablet_manager.cc:1403] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.305s	user 0.018s	sys 0.026s
I20260709 13:06:57.043416 16047 catalog_manager.cc:5697] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.054030 16416 tablet_bootstrap.cc:492] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:57.054543 16416 ts_tablet_manager.cc:1403] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.181s	user 0.027s	sys 0.019s
I20260709 13:06:57.060446 16805 consensus_queue.cc:1048] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:57.063549 16769 leader_election.cc:290] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:57.064239 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:57.065668 16337 leader_election.cc:343] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.076270 16418 ts_tablet_manager.cc:1434] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.121s	user 0.011s	sys 0.005s
W20260709 13:06:57.044999 16188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 74602.6us
I20260709 13:06:57.082909 16416 raft_consensus.cc:359] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:57.083932 16416 raft_consensus.cc:385] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.084276 16416 raft_consensus.cc:740] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.072682 16796 consensus_queue.cc:1048] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:06:57.072875 16807 consensus_queue.cc:1048] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:06:57.087126 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:57.104719 16295 raft_consensus.cc:3060] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.105536 16417 raft_consensus.cc:359] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
W20260709 13:06:57.106272 16338 leader_election.cc:343] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.106941 16338 leader_election.cc:304] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:57.108814 16807 raft_consensus.cc:2749] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.129578 16808 raft_consensus.cc:493] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.130193 16808 raft_consensus.cc:515] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.178606 16299 raft_consensus.cc:3060] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.187714 16416 consensus_queue.cc:260] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:57.190905 16415 tablet_bootstrap.cc:654] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.196978 16416 ts_tablet_manager.cc:1434] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.142s	user 0.011s	sys 0.010s
I20260709 13:06:57.208698 16416 tablet_bootstrap.cc:492] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:57.106323 16417 raft_consensus.cc:385] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.210363 16417 raft_consensus.cc:740] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.211030 16417 consensus_queue.cc:260] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.211882 16808 leader_election.cc:290] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:57.212622 16417 ts_tablet_manager.cc:1434] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.160s	user 0.006s	sys 0.000s
I20260709 13:06:57.213661 16417 tablet_bootstrap.cc:492] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:57.248920 16416 tablet_bootstrap.cc:654] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.258440 16780 raft_consensus.cc:493] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.259207 16780 raft_consensus.cc:515] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.271113 16780 leader_election.cc:290] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
W20260709 13:06:57.288744 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:57.309257 16417 tablet_bootstrap.cc:654] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.314458 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:57.315318 16372 raft_consensus.cc:2468] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:57.316212 16150 raft_consensus.cc:3060] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.317736 16262 leader_election.cc:304] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:57.319041 16780 raft_consensus.cc:2804] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:06:57.319391 16780 raft_consensus.cc:493] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.319772 16780 raft_consensus.cc:3060] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.355995 16418 tablet_bootstrap.cc:492] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:57.376264 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b15e00d79d42809ddd6a31dd9ee9f3" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:57.377264 16297 raft_consensus.cc:2468] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:57.388165 16415 tablet_bootstrap.cc:492] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:57.388901 16415 ts_tablet_manager.cc:1403] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.359s	user 0.018s	sys 0.038s
W20260709 13:06:57.424033 16336 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 66760.6us
I20260709 13:06:57.390872 16365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b15e00d79d42809ddd6a31dd9ee9f3" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:57.433032 16415 raft_consensus.cc:359] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.433879 16415 raft_consensus.cc:385] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.434226 16415 raft_consensus.cc:740] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.435014 16415 consensus_queue.cc:260] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.435997 16365 raft_consensus.cc:2468] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:57.456209 16415 ts_tablet_manager.cc:1434] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:06:57.471033 16416 tablet_bootstrap.cc:492] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:57.471797 16416 ts_tablet_manager.cc:1403] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.263s	user 0.022s	sys 0.030s
I20260709 13:06:57.476532 16417 tablet_bootstrap.cc:492] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:57.477388 16417 ts_tablet_manager.cc:1403] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.264s	user 0.018s	sys 0.007s
I20260709 13:06:57.445024 16760 consensus_queue.cc:1048] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:06:57.480890 16415 tablet_bootstrap.cc:492] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:57.484741 16811 consensus_queue.cc:1048] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:06:57.490159 16416 raft_consensus.cc:359] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.490808 16416 raft_consensus.cc:385] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.491062 16416 raft_consensus.cc:740] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.494304 16780 raft_consensus.cc:515] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.449796 16189 leader_election.cc:304] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:57.496236 16809 raft_consensus.cc:2804] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:06:57.496156 16189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 49908.1us
I20260709 13:06:57.491611 16416 consensus_queue.cc:260] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.518983 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:57.530532 16418 tablet_bootstrap.cc:654] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.499881 16809 raft_consensus.cc:493] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.532346 16809 raft_consensus.cc:3060] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.534969 16298 raft_consensus.cc:1240] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Rejecting Update request from peer 3ce6da6d71124256be0679451014f776 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:06:57.537842 16294 raft_consensus.cc:3060] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.538166 16780 leader_election.cc:290] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:57.539115 16415 tablet_bootstrap.cc:654] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.541460 16417 raft_consensus.cc:359] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:57.542188 16417 raft_consensus.cc:385] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.542486 16417 raft_consensus.cc:740] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.549561 16416 ts_tablet_manager.cc:1434] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.000s
I20260709 13:06:57.559976 16417 consensus_queue.cc:260] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:57.557567 16365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:57.564039 16365 raft_consensus.cc:3060] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.569083 16417 ts_tablet_manager.cc:1434] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.091s	user 0.005s	sys 0.001s
I20260709 13:06:57.572052 16416 tablet_bootstrap.cc:492] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:57.576712 16498 consensus_queue.cc:1059] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:57.577908 16417 tablet_bootstrap.cc:492] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:57.579595 16808 raft_consensus.cc:3055] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:06:57.609807 16416 tablet_bootstrap.cc:654] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.601374 16808 raft_consensus.cc:740] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:57.594091 16365 raft_consensus.cc:2468] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 2.
I20260709 13:06:57.626984 16860 raft_consensus.cc:493] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.641394 16262 leader_election.cc:304] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:57.629386 16860 raft_consensus.cc:515] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:57.618520 16808 consensus_queue.cc:260] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.672323 16808 raft_consensus.cc:3060] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:06:57.655288 16798 raft_consensus.cc:493] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.659415 16415 tablet_bootstrap.cc:492] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:57.675311 16415 ts_tablet_manager.cc:1403] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.206s	user 0.017s	sys 0.001s
I20260709 13:06:57.676812 16760 raft_consensus.cc:493] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.661321 16417 tablet_bootstrap.cc:654] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.669517 16809 raft_consensus.cc:515] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.645571 16796 raft_consensus.cc:2804] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:06:57.674625 16798 raft_consensus.cc:515] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.681929 16796 raft_consensus.cc:697] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:57.682916 16796 consensus_queue.cc:237] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.688622 16418 tablet_bootstrap.cc:492] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:57.689163 16418 ts_tablet_manager.cc:1403] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.339s	user 0.025s	sys 0.014s
I20260709 13:06:57.690587 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dea723251ed4848b42709f873b7d446" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:57.692812 16855 consensus_queue.cc:1048] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:57.691926 16418 raft_consensus.cc:359] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:57.695943 16418 raft_consensus.cc:385] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.696228 16418 raft_consensus.cc:740] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.697199 16418 consensus_queue.cc:260] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:57.702521 16415 raft_consensus.cc:359] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.703572 16415 raft_consensus.cc:385] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.705893 16415 raft_consensus.cc:740] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.677342 16760 raft_consensus.cc:515] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.718055 16418 ts_tablet_manager.cc:1434] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:06:57.739246 16365 raft_consensus.cc:1240] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Rejecting Update request from peer 3ce6da6d71124256be0679451014f776 for earlier term 1. Current term is 2. Ops: []
I20260709 13:06:57.739637 16219 raft_consensus.cc:2410] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 115c2d35eda349178144f8365481e89c for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:06:57.743191 16415 consensus_queue.cc:260] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.747767 16760 leader_election.cc:290] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:57.748131 16798 leader_election.cc:290] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:57.756706 16418 tablet_bootstrap.cc:492] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:57.761314 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c52cafca664c5f90589901f62e4b16" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:57.763872 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26c52cafca664c5f90589901f62e4b16" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:57.765621 16264 leader_election.cc:304] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:57.766909 16365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cbd7d8de354586aab3116154d3ee3c" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:57.769861 16415 ts_tablet_manager.cc:1434] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.094s	user 0.007s	sys 0.000s
I20260709 13:06:57.770944 16415 tablet_bootstrap.cc:492] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:57.784844 16798 raft_consensus.cc:2749] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:06:57.794591 16188 leader_election.cc:343] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.796170 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6959f2cf9714e91aee31bf53776915b" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:57.796790 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6959f2cf9714e91aee31bf53776915b" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:57.797490 16144 raft_consensus.cc:2468] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:57.800581 16860 leader_election.cc:290] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:57.800966 16418 tablet_bootstrap.cc:654] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.802279 16190 leader_election.cc:304] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:06:57.812839 16416 tablet_bootstrap.cc:492] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:57.813529 16416 ts_tablet_manager.cc:1403] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.253s	user 0.024s	sys 0.006s
I20260709 13:06:57.816700 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cbd7d8de354586aab3116154d3ee3c" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
W20260709 13:06:57.817399 16188 leader_election.cc:343] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.801735 16809 leader_election.cc:290] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:57.834898 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b15e00d79d42809ddd6a31dd9ee9f3" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:57.835598 16294 raft_consensus.cc:3060] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.835924 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30b15e00d79d42809ddd6a31dd9ee9f3" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:57.836547 16373 raft_consensus.cc:3060] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.823678 16710 raft_consensus.cc:493] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.838415 16710 raft_consensus.cc:515] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.840490 16710 leader_election.cc:290] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
W20260709 13:06:57.842374 16189 leader_election.cc:343] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.842829 16189 leader_election.cc:304] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:57.843598 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba3cb60361b431ba89f7842cdb1c85a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:57.844869 16416 raft_consensus.cc:359] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.845561 16416 raft_consensus.cc:385] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.843848 16760 raft_consensus.cc:2749] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:57.846748 16416 raft_consensus.cc:740] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.847424 16416 consensus_queue.cc:260] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:06:57.858371 16264 leader_election.cc:343] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.861740 16416 ts_tablet_manager.cc:1434] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 13:06:57.864708 16294 raft_consensus.cc:2468] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:57.812163 16860 raft_consensus.cc:2804] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.880582 16189 leader_election.cc:304] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:57.887370 16416 tablet_bootstrap.cc:492] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:57.881031 16860 raft_consensus.cc:493] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.883869 16809 raft_consensus.cc:2804] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:57.888794 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba3cb60361b431ba89f7842cdb1c85a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:57.896963 16219 raft_consensus.cc:2468] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:57.901270 16415 tablet_bootstrap.cc:654] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.904523 16262 leader_election.cc:304] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:57.908677 16373 raft_consensus.cc:2468] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:57.905470 16771 raft_consensus.cc:2804] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:57.915424 16418 tablet_bootstrap.cc:492] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:57.922508 16418 ts_tablet_manager.cc:1403] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.177s	user 0.017s	sys 0.015s
I20260709 13:06:57.922785 16416 tablet_bootstrap.cc:654] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:57.894372 16860 raft_consensus.cc:3060] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.914160 16771 raft_consensus.cc:493] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:57.936203 16771 raft_consensus.cc:3060] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:57.913499 16809 raft_consensus.cc:697] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:57.939370 16809 consensus_queue.cc:237] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.944412 16771 raft_consensus.cc:515] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.947219 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba3cb60361b431ba89f7842cdb1c85a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
W20260709 13:06:57.948773 16264 leader_election.cc:343] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.923709 16048 catalog_manager.cc:5697] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 1 to 2, leader changed from 3ce6da6d71124256be0679451014f776 (127.7.213.66) to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 2 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:57.925901 16417 tablet_bootstrap.cc:492] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:57.950605 16417 ts_tablet_manager.cc:1403] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.373s	user 0.041s	sys 0.031s
I20260709 13:06:57.929987 16418 raft_consensus.cc:359] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:57.952742 16418 raft_consensus.cc:385] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.953058 16418 raft_consensus.cc:740] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.935276 16416 tablet_bootstrap.cc:492] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:57.953927 16416 ts_tablet_manager.cc:1403] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.067s	user 0.009s	sys 0.003s
I20260709 13:06:57.958837 16418 consensus_queue.cc:260] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:57.964365 16417 raft_consensus.cc:359] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.965149 16417 raft_consensus.cc:385] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.965412 16417 raft_consensus.cc:740] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.966109 16417 consensus_queue.cc:260] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:57.984433 16417 ts_tablet_manager.cc:1434] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:06:57.985893 16417 tablet_bootstrap.cc:492] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964: Bootstrap starting.
I20260709 13:06:57.986104 16772 raft_consensus.cc:493] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:57.986948 16772 raft_consensus.cc:515] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:57.989827 16416 raft_consensus.cc:359] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.990847 16416 raft_consensus.cc:385] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:57.991182 16416 raft_consensus.cc:740] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:57.992746 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:57.992945 16771 leader_election.cc:290] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:57.993876 16416 consensus_queue.cc:260] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:57.998994 16418 ts_tablet_manager.cc:1434] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.076s	user 0.002s	sys 0.004s
I20260709 13:06:58.006762 16416 ts_tablet_manager.cc:1434] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.053s	user 0.003s	sys 0.005s
I20260709 13:06:57.995645 16772 leader_election.cc:290] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
W20260709 13:06:58.013763 16338 leader_election.cc:343] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:57.995015 16860 raft_consensus.cc:515] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.004179 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba3cb60361b431ba89f7842cdb1c85a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:58.015615 16416 tablet_bootstrap.cc:492] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776: Bootstrap starting.
I20260709 13:06:58.015646 16418 tablet_bootstrap.cc:492] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:58.017894 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:06:58.022598 16336 leader_election.cc:343] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.023034 16336 leader_election.cc:304] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:58.023736 16851 raft_consensus.cc:2749] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.019233 16220 raft_consensus.cc:3060] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.024431 16851 raft_consensus.cc:493] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.024806 16416 tablet_bootstrap.cc:654] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.024868 16851 raft_consensus.cc:515] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:58.028483 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cbd7d8de354586aab3116154d3ee3c" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:58.029079 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cbd7d8de354586aab3116154d3ee3c" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:58.029727 16219 raft_consensus.cc:2468] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
W20260709 13:06:58.029960 16337 leader_election.cc:343] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.031025 16336 leader_election.cc:304] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:58.032552 16851 leader_election.cc:290] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:58.032773 16848 raft_consensus.cc:2804] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.033376 16848 raft_consensus.cc:493] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.033759 16848 raft_consensus.cc:3060] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.074831 16047 catalog_manager.cc:5697] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.090023 16860 leader_election.cc:290] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:58.091871 16814 raft_consensus.cc:493] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.092463 16814 raft_consensus.cc:515] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.093319 16415 tablet_bootstrap.cc:492] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:58.093860 16415 ts_tablet_manager.cc:1403] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.323s	user 0.041s	sys 0.034s
I20260709 13:06:58.094059 16365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6959f2cf9714e91aee31bf53776915b" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:58.094785 16814 leader_election.cc:290] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:58.096920 16415 raft_consensus.cc:359] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.097700 16415 raft_consensus.cc:385] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.097990 16415 raft_consensus.cc:740] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.098469 16365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bbd6b3bdff467f9d48329073cf0167" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:58.098755 16415 consensus_queue.cc:260] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.105211 16415 ts_tablet_manager.cc:1434] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:06:58.106389 16415 tablet_bootstrap.cc:492] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
W20260709 13:06:58.108860 16188 leader_election.cc:343] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.099486 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "38bbd6b3bdff467f9d48329073cf0167" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:58.122502 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6959f2cf9714e91aee31bf53776915b" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:58.123193 16150 raft_consensus.cc:3060] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.123405 16114 leader_election.cc:304] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:06:58.125556 16619 raft_consensus.cc:2749] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.130801 16220 raft_consensus.cc:2468] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:58.132802 16365 raft_consensus.cc:1275] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer 115c2d35eda349178144f8365481e89c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:06:58.137934 16417 tablet_bootstrap.cc:654] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.141862 16262 leader_election.cc:304] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:58.143841 16150 raft_consensus.cc:2468] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:58.144917 16796 consensus_queue.cc:1048] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.149401 16190 leader_election.cc:304] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: c8106927ed954e1c800869427b6c7964
I20260709 13:06:58.150714 16809 raft_consensus.cc:2804] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.151283 16809 raft_consensus.cc:697] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:58.152225 16809 consensus_queue.cc:237] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.154647 16418 tablet_bootstrap.cc:654] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.155210 16604 raft_consensus.cc:2804] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.157114 16604 raft_consensus.cc:697] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:58.165660 16604 consensus_queue.cc:237] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.167359 16416 tablet_bootstrap.cc:492] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776: No bootstrap required, opened a new log
I20260709 13:06:58.167996 16416 ts_tablet_manager.cc:1403] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776: Time spent bootstrapping tablet: real 0.155s	user 0.027s	sys 0.044s
I20260709 13:06:58.172233 16141 raft_consensus.cc:3060] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.155881 16848 raft_consensus.cc:515] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:58.205219 16416 raft_consensus.cc:359] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.206212 16416 raft_consensus.cc:385] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.209810 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cbd7d8de354586aab3116154d3ee3c" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:58.210481 16220 raft_consensus.cc:3060] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.206565 16416 raft_consensus.cc:740] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.217969 16416 consensus_queue.cc:260] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.236029 16416 ts_tablet_manager.cc:1434] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.001s
I20260709 13:06:58.228469 16769 raft_consensus.cc:493] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.232921 16848 leader_election.cc:290] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:58.243188 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cbd7d8de354586aab3116154d3ee3c" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:58.248574 16769 raft_consensus.cc:515] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.256104 16769 leader_election.cc:290] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:06:58.268735 16337 leader_election.cc:343] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.270133 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:58.272531 16299 raft_consensus.cc:2468] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:58.264879 16220 raft_consensus.cc:2468] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:58.274511 16415 tablet_bootstrap.cc:654] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.277319 16337 leader_election.cc:304] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:58.285422 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:58.286319 16148 raft_consensus.cc:2468] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 0.
I20260709 13:06:58.310222 16218 raft_consensus.cc:1275] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer 115c2d35eda349178144f8365481e89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:06:58.310771 16218 pending_rounds.cc:85] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776: Aborting all ops after (but not including) 0
I20260709 13:06:58.310995 16218 pending_rounds.cc:107] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:06:58.311304 16218 raft_consensus.cc:2889] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:06:58.313730 16417 tablet_bootstrap.cc:492] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964: No bootstrap required, opened a new log
I20260709 13:06:58.314248 16417 ts_tablet_manager.cc:1403] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964: Time spent bootstrapping tablet: real 0.329s	user 0.031s	sys 0.036s
I20260709 13:06:58.296510 16860 raft_consensus.cc:493] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.317176 16860 raft_consensus.cc:515] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.287485 16848 raft_consensus.cc:2804] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.318558 16848 raft_consensus.cc:493] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.322548 16336 leader_election.cc:304] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:58.332954 16048 catalog_manager.cc:5697] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.324620 16848 raft_consensus.cc:3060] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.343161 16860 leader_election.cc:290] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:58.343866 16418 tablet_bootstrap.cc:492] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:58.344374 16418 ts_tablet_manager.cc:1403] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.336s	user 0.027s	sys 0.016s
I20260709 13:06:58.345907 16365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:58.346589 16365 raft_consensus.cc:2468] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 0.
I20260709 13:06:58.347787 16188 leader_election.cc:304] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:58.348631 16809 raft_consensus.cc:2804] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.348985 16809 raft_consensus.cc:493] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.349301 16809 raft_consensus.cc:3060] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.351375 16417 raft_consensus.cc:359] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.352103 16417 raft_consensus.cc:385] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.352358 16417 raft_consensus.cc:740] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.353188 16417 consensus_queue.cc:260] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.356602 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:58.358090 16190 leader_election.cc:343] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.330315 16851 raft_consensus.cc:2804] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.358952 16851 raft_consensus.cc:697] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:58.355698 16417 ts_tablet_manager.cc:1434] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.000s
I20260709 13:06:58.359577 16418 raft_consensus.cc:359] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.372654 16418 raft_consensus.cc:385] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:06:58.372223 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 90953.7us
I20260709 13:06:58.372994 16418 raft_consensus.cc:740] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.373715 16418 consensus_queue.cc:260] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.363189 16851 consensus_queue.cc:237] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:58.384670 16848 raft_consensus.cc:515] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.387104 16848 leader_election.cc:290] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:58.388269 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:58.389038 16148 raft_consensus.cc:3060] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.395295 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:58.396391 16418 ts_tablet_manager.cc:1434] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.000s
I20260709 13:06:58.402248 16148 raft_consensus.cc:2468] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:58.403389 16338 leader_election.cc:304] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:06:58.412201 16045 catalog_manager.cc:5697] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.417124 16418 tablet_bootstrap.cc:492] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:58.414736 16798 consensus_queue.cc:1048] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:06:58.411875 16299 raft_consensus.cc:3060] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.415148 16884 raft_consensus.cc:2804] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.425743 16884 raft_consensus.cc:697] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:06:58.426607 16884 consensus_queue.cc:237] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:06:58.430605 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:06:58.436954 16299 raft_consensus.cc:2468] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:06:58.432376 16760 raft_consensus.cc:493] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.434279 16860 consensus_queue.cc:1048] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.445771 16760 raft_consensus.cc:515] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.468294 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714e2c053f14fbeaec12683da401eb4" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:06:58.476732 16190 leader_election.cc:343] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.489883 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714e2c053f14fbeaec12683da401eb4" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:58.491061 16772 raft_consensus.cc:493] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.491703 16772 raft_consensus.cc:515] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.494872 16760 leader_election.cc:290] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:58.494968 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6959f2cf9714e91aee31bf53776915b" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:06:58.495234 16189 leader_election.cc:343] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.497115 16189 leader_election.cc:304] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:58.497318 16809 raft_consensus.cc:515] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.499431 16809 leader_election.cc:290] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:06:58.499480 16498 raft_consensus.cc:2749] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.501690 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:58.503578 16418 tablet_bootstrap.cc:654] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.505221 16498 consensus_queue.cc:1048] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.506927 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:58.507586 16373 raft_consensus.cc:3060] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.510300 16415 tablet_bootstrap.cc:492] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:58.510789 16415 ts_tablet_manager.cc:1403] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.405s	user 0.027s	sys 0.018s
W20260709 13:06:58.512091 16190 leader_election.cc:343] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.513441 16415 raft_consensus.cc:359] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.514111 16415 raft_consensus.cc:385] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.514358 16415 raft_consensus.cc:740] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.514977 16415 consensus_queue.cc:260] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.517490 16415 ts_tablet_manager.cc:1434] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:58.517985 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6959f2cf9714e91aee31bf53776915b" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:58.518420 16415 tablet_bootstrap.cc:492] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:58.518981 16148 raft_consensus.cc:2393] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 1: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
I20260709 13:06:58.527449 16338 leader_election.cc:304] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:06:58.530040 16769 raft_consensus.cc:3060] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.531409 16415 tablet_bootstrap.cc:654] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.533488 16373 raft_consensus.cc:2468] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:06:58.548012 16188 leader_election.cc:304] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
W20260709 13:06:58.549430 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 107898us
I20260709 13:06:58.552539 16860 raft_consensus.cc:2804] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.553154 16860 raft_consensus.cc:697] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:06:58.554098 16860 consensus_queue.cc:237] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:58.556891 16415 tablet_bootstrap.cc:492] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:58.557435 16415 ts_tablet_manager.cc:1403] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.002s
I20260709 13:06:58.561192 16415 raft_consensus.cc:359] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:58.561887 16415 raft_consensus.cc:385] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.562167 16415 raft_consensus.cc:740] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.562873 16415 consensus_queue.cc:260] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:58.568442 16818 raft_consensus.cc:493] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.568984 16818 raft_consensus.cc:515] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.571019 16818 leader_election.cc:290] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:58.573544 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:58.574791 16772 leader_election.cc:290] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
W20260709 13:06:58.575050 16114 leader_election.cc:343] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.575969 16366 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:58.576658 16366 raft_consensus.cc:2468] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 0.
I20260709 13:06:58.577874 16113 leader_election.cc:304] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:58.579326 16818 raft_consensus.cc:2804] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.580121 16818 raft_consensus.cc:493] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.580569 16818 raft_consensus.cc:3060] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.588153 16415 ts_tablet_manager.cc:1434] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:06:58.596812 16415 tablet_bootstrap.cc:492] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c: Bootstrap starting.
I20260709 13:06:58.542224 16047 catalog_manager.cc:5697] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.611801 16818 raft_consensus.cc:515] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:06:58.649065 16901 log.cc:927] Time spent T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:06:58.662813 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:06:58.663252 16818 leader_election.cc:290] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:58.664479 16365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:06:58.665812 16365 raft_consensus.cc:3060] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:06:58.664673 16114 leader_election.cc:343] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.668288 16769 raft_consensus.cc:2749] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.674710 16047 catalog_manager.cc:5697] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.676429 16809 consensus_queue.cc:1048] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.680882 16415 tablet_bootstrap.cc:654] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.685228 16365 raft_consensus.cc:2468] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 1.
I20260709 13:06:58.686589 16113 leader_election.cc:304] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c
I20260709 13:06:58.687948 16818 raft_consensus.cc:2804] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.688685 16818 raft_consensus.cc:697] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:06:58.691854 16818 consensus_queue.cc:237] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.719120 16418 tablet_bootstrap.cc:492] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:58.719620 16418 ts_tablet_manager.cc:1403] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.303s	user 0.027s	sys 0.033s
I20260709 13:06:58.722841 16796 consensus_queue.cc:1048] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.719416 16044 catalog_manager.cc:5697] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 1 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:58.728366 16418 raft_consensus.cc:359] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.729354 16418 raft_consensus.cc:385] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.729713 16418 raft_consensus.cc:740] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.748211 16418 consensus_queue.cc:260] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.752031 16150 raft_consensus.cc:3060] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.753103 16418 ts_tablet_manager.cc:1434] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.001s
I20260709 13:06:58.768041 16418 tablet_bootstrap.cc:492] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:58.770077 16415 tablet_bootstrap.cc:492] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c: No bootstrap required, opened a new log
I20260709 13:06:58.770740 16415 ts_tablet_manager.cc:1403] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c: Time spent bootstrapping tablet: real 0.174s	user 0.038s	sys 0.021s
I20260709 13:06:58.773859 16415 raft_consensus.cc:359] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.774581 16415 raft_consensus.cc:385] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.774799 16418 tablet_bootstrap.cc:654] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.774981 16415 raft_consensus.cc:740] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.775822 16415 consensus_queue.cc:260] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:06:58.776777 16796 raft_consensus.cc:493] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.777204 16796 raft_consensus.cc:515] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.779157 16796 leader_election.cc:290] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:58.780853 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714e2c053f14fbeaec12683da401eb4" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:58.781594 16219 raft_consensus.cc:2468] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 0.
I20260709 13:06:58.783011 16262 leader_election.cc:304] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:58.783594 16855 consensus_queue.cc:1048] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.783938 16796 raft_consensus.cc:2804] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:06:58.784554 16796 raft_consensus.cc:493] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:06:58.784981 16796 raft_consensus.cc:3060] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.788890 16044 catalog_manager.cc:5697] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 1 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:06:58.808933 16796 raft_consensus.cc:515] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.811110 16796 leader_election.cc:290] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:58.812409 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714e2c053f14fbeaec12683da401eb4" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:06:58.817706 16798 raft_consensus.cc:493] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.819234 16219 raft_consensus.cc:3060] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.818248 16798 raft_consensus.cc:515] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:06:58.814702 16892 consensus_queue.cc:1048] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:58.815299 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714e2c053f14fbeaec12683da401eb4" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
I20260709 13:06:58.822381 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cbd7d8de354586aab3116154d3ee3c" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:58.816044 16141 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714e2c053f14fbeaec12683da401eb4" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
I20260709 13:06:58.823156 16220 raft_consensus.cc:2393] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 115c2d35eda349178144f8365481e89c in current term 1: Already voted for candidate c8106927ed954e1c800869427b6c7964 in this term.
I20260709 13:06:58.829403 16415 ts_tablet_manager.cc:1434] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.000s
I20260709 13:06:58.835379 16219 raft_consensus.cc:2468] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:06:58.837468 16262 leader_election.cc:304] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:06:58.845177 16798 leader_election.cc:290] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:06:58.845820 16798 raft_consensus.cc:2804] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:06:58.849045 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cbd7d8de354586aab3116154d3ee3c" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
W20260709 13:06:58.849470 16264 leader_election.cc:343] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:06:58.850435 16264 leader_election.cc:343] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:06:58.851348 16262 leader_election.cc:304] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:58.853216 16771 raft_consensus.cc:3060] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.853696 16798 raft_consensus.cc:697] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:06:58.854641 16798 consensus_queue.cc:237] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.867240 16771 raft_consensus.cc:2749] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:58.869351 16418 tablet_bootstrap.cc:492] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:58.870101 16418 ts_tablet_manager.cc:1403] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.102s	user 0.016s	sys 0.035s
I20260709 13:06:58.872790 16418 raft_consensus.cc:359] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.873489 16418 raft_consensus.cc:385] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:58.873749 16418 raft_consensus.cc:740] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:58.877967 16418 consensus_queue.cc:260] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.887050 16418 ts_tablet_manager.cc:1434] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 13:06:58.887476 16772 consensus_queue.cc:1048] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.888273 16418 tablet_bootstrap.cc:492] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:06:58.907261 16418 tablet_bootstrap.cc:654] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: Neither blocks nor log segments found. Creating new log.
I20260709 13:06:58.931890 16851 consensus_queue.cc:1048] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:58.973896 16818 raft_consensus.cc:493] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:58.974501 16818 raft_consensus.cc:515] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:06:58.978924 16818 leader_election.cc:290] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:06:58.980615 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714e2c053f14fbeaec12683da401eb4" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:06:58.983495 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6714e2c053f14fbeaec12683da401eb4" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:58.984351 16219 raft_consensus.cc:2393] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aa8eb07867e7420b80a7e615d570509d in current term 1: Already voted for candidate 115c2d35eda349178144f8365481e89c in this term.
I20260709 13:06:58.986260 16113 leader_election.cc:304] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:06:58.991961 16818 raft_consensus.cc:3060] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:58.998281 16760 consensus_queue.cc:1048] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.017007 16772 consensus_queue.cc:1048] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:59.046413 16190 consensus_peers.cc:597] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:06:59.048077 16418 tablet_bootstrap.cc:492] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: No bootstrap required, opened a new log
I20260709 13:06:59.048640 16418 ts_tablet_manager.cc:1403] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.161s	user 0.025s	sys 0.034s
I20260709 13:06:59.051343 16418 raft_consensus.cc:359] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:59.052052 16418 raft_consensus.cc:385] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:06:59.052328 16418 raft_consensus.cc:740] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:06:59.053120 16418 consensus_queue.cc:260] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:59.055387 16418 ts_tablet_manager.cc:1434] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:06:59.093988 16294 raft_consensus.cc:3060] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:06:59.138511 16772 consensus_queue.cc:1048] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:06:59.150553 16113 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 44047.5us
I20260709 13:06:59.161718 16045 catalog_manager.cc:5697] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 0 to 1, leader changed from <none> to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 1 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260709 13:06:59.169348 16818 raft_consensus.cc:2749] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:59.179836 16589 consensus_queue.cc:1048] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:59.188390 16589 consensus_queue.cc:1048] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.269134 16887 consensus_queue.cc:1048] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:06:59.284898 16771 consensus_queue.cc:1048] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:06:59.552371 16589 raft_consensus.cc:493] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:06:59.552896 16589 raft_consensus.cc:515] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:06:59.554587 16589 leader_election.cc:290] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:06:59.555330 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:06:59.555553 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:06:59.557305 16113 leader_election.cc:304] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:06:59.558162 16589 raft_consensus.cc:2749] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:06:59.560102 16150 raft_consensus.cc:3060] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:00.241801  8021 tablet_server.cc:179] TabletServer@127.7.213.65:0 shutting down...
W20260709 13:07:00.291203 16264 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 555 similar messages]
W20260709 13:07:00.348268 16264 consensus_peers.cc:597] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.349607 16264 consensus_peers.cc:597] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.356330 16264 consensus_peers.cc:597] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.362284 16190 consensus_peers.cc:597] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.360725 16338 consensus_peers.cc:597] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.364352 16264 consensus_peers.cc:597] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.377861 16264 consensus_peers.cc:597] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.378721 16190 consensus_peers.cc:597] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.400393 16264 consensus_peers.cc:597] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.402920 16338 consensus_peers.cc:597] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.421630 16190 consensus_peers.cc:597] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.444828 16264 consensus_peers.cc:597] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.458482 16190 consensus_peers.cc:597] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.465816 16338 consensus_peers.cc:597] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.468606 16264 consensus_peers.cc:597] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.481642 16264 consensus_peers.cc:597] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.484501 16190 consensus_peers.cc:597] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.486378 16338 consensus_peers.cc:597] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.500607 16338 consensus_peers.cc:597] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.518515  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:00.520390  8021 tablet_replica.cc:333] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.522185  8021 raft_consensus.cc:2243] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.523409  8021 raft_consensus.cc:2272] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.528357  8021 tablet_replica.cc:333] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.529655  8021 raft_consensus.cc:2243] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.530673  8021 raft_consensus.cc:2272] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.545789 16190 consensus_peers.cc:597] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:00.549592 16264 consensus_peers.cc:597] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.553383  8021 tablet_replica.cc:333] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.555096  8021 raft_consensus.cc:2243] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.556038  8021 raft_consensus.cc:2272] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.561739 16338 consensus_peers.cc:597] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.567624  8021 tablet_replica.cc:333] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.569379  8021 raft_consensus.cc:2243] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.570261  8021 raft_consensus.cc:2272] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.573685 16190 consensus_peers.cc:597] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.575323  8021 tablet_replica.cc:333] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.579573  8021 raft_consensus.cc:2243] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.580426  8021 raft_consensus.cc:2272] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.583817  8021 tablet_replica.cc:333] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.584923  8021 raft_consensus.cc:2243] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.585662  8021 raft_consensus.cc:2272] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.589471  8021 tablet_replica.cc:333] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
W20260709 13:07:00.591070 16264 consensus_peers.cc:597] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.592408  8021 raft_consensus.cc:2243] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.593303  8021 raft_consensus.cc:2272] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.597640  8021 tablet_replica.cc:333] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.598644  8021 raft_consensus.cc:2243] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.600405  8021 raft_consensus.cc:2272] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.602836 16190 consensus_peers.cc:597] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.603819  8021 tablet_replica.cc:333] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.604866  8021 raft_consensus.cc:2243] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.609769  8021 raft_consensus.cc:2272] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.613098 16190 consensus_peers.cc:597] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.613561  8021 tablet_replica.cc:333] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.618536  8021 raft_consensus.cc:2243] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.619388  8021 raft_consensus.cc:2272] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.623813  8021 tablet_replica.cc:333] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.624794  8021 raft_consensus.cc:2243] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.625607  8021 raft_consensus.cc:2272] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.629339  8021 tablet_replica.cc:333] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
W20260709 13:07:00.632496 16190 consensus_peers.cc:597] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.630489  8021 raft_consensus.cc:2243] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.634917  8021 raft_consensus.cc:2272] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.637131 16190 consensus_peers.cc:597] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.645388  8021 tablet_replica.cc:333] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.646487  8021 raft_consensus.cc:2243] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:00.646788 16190 consensus_peers.cc:597] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.648582  8021 raft_consensus.cc:2272] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.652791  8021 tablet_replica.cc:333] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.654028  8021 raft_consensus.cc:2243] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.654841  8021 raft_consensus.cc:2272] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.658481  8021 tablet_replica.cc:333] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.659446  8021 raft_consensus.cc:2243] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.660859  8021 raft_consensus.cc:2272] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.664525  8021 tablet_replica.cc:333] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.665565  8021 raft_consensus.cc:2243] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.666888  8021 raft_consensus.cc:2272] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.672794 16264 consensus_peers.cc:597] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.674273  8021 tablet_replica.cc:333] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.675508  8021 raft_consensus.cc:2243] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.676884  8021 raft_consensus.cc:2272] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.680857  8021 tablet_replica.cc:333] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.681960  8021 raft_consensus.cc:2243] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.682642  8021 raft_consensus.cc:2272] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.686277  8021 tablet_replica.cc:333] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.687253  8021 raft_consensus.cc:2243] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.688064  8021 raft_consensus.cc:2272] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.689644 16338 consensus_peers.cc:597] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.691802  8021 tablet_replica.cc:333] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.693121  8021 raft_consensus.cc:2243] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.693969  8021 raft_consensus.cc:2272] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.695850 16264 consensus_peers.cc:597] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.698261  8021 tablet_replica.cc:333] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.699604  8021 raft_consensus.cc:2243] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.702283  8021 raft_consensus.cc:2272] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.706025  8021 tablet_replica.cc:333] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.707149  8021 raft_consensus.cc:2243] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.708873  8021 raft_consensus.cc:2272] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.712777  8021 tablet_replica.cc:333] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.713761  8021 raft_consensus.cc:2243] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.715790  8021 raft_consensus.cc:2272] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.717840 16264 consensus_peers.cc:597] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.719430  8021 tablet_replica.cc:333] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.720546  8021 raft_consensus.cc:2243] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.721998  8021 raft_consensus.cc:2272] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.725234  8021 tablet_replica.cc:333] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.726212  8021 raft_consensus.cc:2243] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.726897  8021 raft_consensus.cc:2272] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.731284  8021 tablet_replica.cc:333] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.732425  8021 raft_consensus.cc:2243] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.734207  8021 raft_consensus.cc:2272] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.736560  8021 tablet_replica.cc:333] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.737206  8021 raft_consensus.cc:2243] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.738138  8021 raft_consensus.cc:2272] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.741266  8021 tablet_replica.cc:333] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.742101  8021 raft_consensus.cc:2243] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.742975  8021 raft_consensus.cc:2272] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.745814  8021 tablet_replica.cc:333] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.746752  8021 raft_consensus.cc:2243] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.747860  8021 raft_consensus.cc:2272] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.751044  8021 tablet_replica.cc:333] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.751931  8021 raft_consensus.cc:2243] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.752633  8021 raft_consensus.cc:2272] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:00.751559 16190 consensus_peers.cc:597] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.755548  8021 tablet_replica.cc:333] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
W20260709 13:07:00.756057 16338 consensus_peers.cc:597] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:00.756217  8021 raft_consensus.cc:2243] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.757014  8021 raft_consensus.cc:2272] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.759243  8021 tablet_replica.cc:333] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.759991  8021 raft_consensus.cc:2243] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.760578  8021 raft_consensus.cc:2272] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.763690  8021 tablet_replica.cc:333] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.764524  8021 raft_consensus.cc:2243] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.765213  8021 raft_consensus.cc:2272] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.767664  8021 tablet_replica.cc:333] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.768255  8021 raft_consensus.cc:2243] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.768769  8021 raft_consensus.cc:2272] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.771720  8021 tablet_replica.cc:333] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.772392  8021 raft_consensus.cc:2243] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.772998  8021 raft_consensus.cc:2272] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.776031  8021 tablet_replica.cc:333] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.776629  8021 raft_consensus.cc:2243] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.777252  8021 raft_consensus.cc:2272] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.780154  8021 tablet_replica.cc:333] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.780830  8021 raft_consensus.cc:2243] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.781417  8021 raft_consensus.cc:2272] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.784116  8021 tablet_replica.cc:333] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.784996  8021 raft_consensus.cc:2243] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.785633  8021 raft_consensus.cc:2272] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.788689  8021 tablet_replica.cc:333] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.789206  8021 raft_consensus.cc:2243] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.789799  8021 raft_consensus.cc:2272] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.792495  8021 tablet_replica.cc:333] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.793191  8021 raft_consensus.cc:2243] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.793830  8021 raft_consensus.cc:2272] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.796896  8021 tablet_replica.cc:333] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.797588  8021 raft_consensus.cc:2243] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.798252  8021 raft_consensus.cc:2272] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.801191  8021 tablet_replica.cc:333] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.801916  8021 raft_consensus.cc:2243] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.802646  8021 raft_consensus.cc:2272] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.815138  8021 tablet_replica.cc:333] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.815918  8021 raft_consensus.cc:2243] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:00.816612  8021 raft_consensus.cc:2272] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.819231  8021 tablet_replica.cc:333] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.819972  8021 raft_consensus.cc:2243] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.821070  8021 raft_consensus.cc:2272] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.824489  8021 tablet_replica.cc:333] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: stopping tablet replica
I20260709 13:07:00.825179  8021 raft_consensus.cc:2243] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:00.826400  8021 raft_consensus.cc:2272] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:00.931984  8021 tablet_server.cc:196] TabletServer@127.7.213.65:0 shutdown complete.
I20260709 13:07:01.076853  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:01.151659 16940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:01.162955 16943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:01.170159  8021 server_base.cc:1061] running on GCE node
W20260709 13:07:01.193683 16938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:01.202957  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:01.203372  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:01.203626  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602421203606 us; error 0 us; skew 500 ppm
I20260709 13:07:01.204464  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:01.219928  8021 webserver.cc:533] Webserver started at http://127.7.213.65:38325/ using document root <none> and password file <none>
I20260709 13:07:01.220727  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:01.221009  8021 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:07:01.229370  8021 fs_manager.cc:714] Time spent opening directory manager: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:07:01.244769 16949 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:01.248206  8021 fs_manager.cc:730] Time spent opening block manager: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:07:01.248656  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "aa8eb07867e7420b80a7e615d570509d"
format_stamp: "Formatted at 2026-07-09 13:06:46 on dist-test-slave-9g29"
I20260709 13:07:01.249938  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:01.304978  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:01.307011  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:01.313481  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:01.320016 16956 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:07:01.532936 16771 raft_consensus.cc:493] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.533948 16771 raft_consensus.cc:515] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:01.536556 16771 leader_election.cc:290] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:01.545679 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2201023e16254eed9712e5fba24caa38" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:07:01.564944 16264 leader_election.cc:336] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.565454 16264 leader_election.cc:304] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:01.566216 16771 raft_consensus.cc:2749] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:01.600452  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:07:01.600957  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.282s	user 0.009s	sys 0.001s
I20260709 13:07:01.601441  8021 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:07:01.609736 16956 tablet_bootstrap.cc:492] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:01.628445 16956 tablet_bootstrap.cc:492] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:01.629456 16956 tablet_bootstrap.cc:492] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:01.630168 16956 ts_tablet_manager.cc:1403] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 13:07:01.632988 16956 raft_consensus.cc:359] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:01.633739 16956 raft_consensus.cc:740] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.634536 16956 consensus_queue.cc:260] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:01.640972 16956 ts_tablet_manager.cc:1434] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:07:01.641901 16956 tablet_bootstrap.cc:492] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:01.676229 16956 tablet_bootstrap.cc:492] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:01.680285 16956 tablet_bootstrap.cc:492] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:01.681293 16956 ts_tablet_manager.cc:1403] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.008s
I20260709 13:07:01.688716 16956 raft_consensus.cc:359] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:01.689636 16956 raft_consensus.cc:740] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.690497 16956 consensus_queue.cc:260] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:01.692512 16956 ts_tablet_manager.cc:1434] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:07:01.693333 16956 tablet_bootstrap.cc:492] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:01.754984 16956 tablet_bootstrap.cc:492] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:01.756012 16956 tablet_bootstrap.cc:492] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:01.756696 16956 ts_tablet_manager.cc:1403] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.028s
I20260709 13:07:01.759233 16956 raft_consensus.cc:359] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.759950 16956 raft_consensus.cc:740] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.760870 16956 consensus_queue.cc:260] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.762960 16956 ts_tablet_manager.cc:1434] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:01.763716 16956 tablet_bootstrap.cc:492] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:01.770398 16937 raft_consensus.cc:493] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.770995 16937 raft_consensus.cc:515] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.775645 16892 raft_consensus.cc:493] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.776105 16937 leader_election.cc:290] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:01.776538 16892 raft_consensus.cc:515] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.792344 16956 tablet_bootstrap.cc:492] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:01.793433 16956 tablet_bootstrap.cc:492] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:01.794083 16956 ts_tablet_manager.cc:1403] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.012s
I20260709 13:07:01.794101 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
W20260709 13:07:01.794757 16264 leader_election.cc:336] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.794898 16373 raft_consensus.cc:2468] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:07:01.796728 16956 raft_consensus.cc:359] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.797420 16956 raft_consensus.cc:740] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.798276 16956 consensus_queue.cc:260] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.800450 16956 ts_tablet_manager.cc:1434] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:07:01.800623 16892 leader_election.cc:290] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:01.801222 16956 tablet_bootstrap.cc:492] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
W20260709 13:07:01.801566 16338 leader_election.cc:336] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.802620 16262 leader_election.cc:304] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:01.804940 16937 raft_consensus.cc:493] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.808806 16937 raft_consensus.cc:515] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.804472 16892 raft_consensus.cc:2804] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:01.812026 16892 raft_consensus.cc:493] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.812418 16892 raft_consensus.cc:3060] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:01.815560 16937 leader_election.cc:290] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:07:01.822461 16338 leader_election.cc:336] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.829864 16956 tablet_bootstrap.cc:492] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:01.830996 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:01.835140 16956 tablet_bootstrap.cc:492] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:01.835949 16956 ts_tablet_manager.cc:1403] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.035s	user 0.023s	sys 0.000s
I20260709 13:07:01.838138 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:01.838598 16956 raft_consensus.cc:359] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:01.839355 16956 raft_consensus.cc:740] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.840157 16956 consensus_queue.cc:260] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:01.852205 16956 ts_tablet_manager.cc:1434] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:07:01.853013 16956 tablet_bootstrap.cc:492] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:01.865101 16892 raft_consensus.cc:515] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.866046 16962 raft_consensus.cc:493] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.866616 16962 raft_consensus.cc:515] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.868232 16294 raft_consensus.cc:2393] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c8106927ed954e1c800869427b6c7964 in current term 2: Already voted for candidate 115c2d35eda349178144f8365481e89c in this term.
I20260709 13:07:01.872502 16962 leader_election.cc:290] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:01.874117 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:01.877351 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6563b9b2e954646b0ce2535138676d0" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:01.877923 16370 raft_consensus.cc:3060] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:01.879254 16337 leader_election.cc:304] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:01.880190 16929 raft_consensus.cc:2749] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:01.882750 16956 tablet_bootstrap.cc:492] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:01.882598 16337 leader_election.cc:304] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:01.883827 16956 tablet_bootstrap.cc:492] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:01.883934 16962 raft_consensus.cc:493] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.884573 16956 ts_tablet_manager.cc:1403] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.007s
I20260709 13:07:01.888019 16956 raft_consensus.cc:359] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:01.888379 16926 raft_consensus.cc:493] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.888803 16956 raft_consensus.cc:740] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.889276 16892 leader_election.cc:290] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:01.888979 16926 raft_consensus.cc:515] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:01.888420 16370 raft_consensus.cc:2468] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 2.
I20260709 13:07:01.889789 16956 consensus_queue.cc:260] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:01.893524 16929 raft_consensus.cc:2749] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:01.884522 16962 raft_consensus.cc:515] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.895280 16262 leader_election.cc:304] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
W20260709 13:07:01.898398 16190 leader_election.cc:336] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.899112 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2201023e16254eed9712e5fba24caa38" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:01.903378 16956 ts_tablet_manager.cc:1434] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 13:07:01.904431 16956 tablet_bootstrap.cc:492] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:01.905431 16892 raft_consensus.cc:2804] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:01.906450  8021 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:07:01.906831  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.305s	user 0.258s	sys 0.014s
I20260709 13:07:01.907133 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:01.891361 16926 leader_election.cc:290] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:07:01.903362 16294 raft_consensus.cc:2468] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:07:01.909253 16962 leader_election.cc:290] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:01.909363 16189 leader_election.cc:304] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:01.909955 16892 raft_consensus.cc:697] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:01.910827 16925 raft_consensus.cc:2804] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:01.911389 16925 raft_consensus.cc:493] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.911187 16892 consensus_queue.cc:237] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.911931 16925 raft_consensus.cc:3060] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:01.917017 16264 leader_election.cc:336] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.918097 16264 leader_election.cc:304] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:07:01.921402 16962 raft_consensus.cc:2749] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:01.923166 16925 raft_consensus.cc:515] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:07:01.924067 16264 leader_election.cc:336] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.927953 16964 raft_consensus.cc:493] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.928512 16956 tablet_bootstrap.cc:492] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:01.928521 16964 raft_consensus.cc:515] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.929651 16956 tablet_bootstrap.cc:492] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:01.930334 16956 ts_tablet_manager.cc:1403] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.008s
I20260709 13:07:01.933334 16956 raft_consensus.cc:359] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:01.933986 16956 raft_consensus.cc:740] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.934254 16964 leader_election.cc:290] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:01.934651 16956 consensus_queue.cc:260] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
W20260709 13:07:01.934978 16264 leader_election.cc:336] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.935549 16264 leader_election.cc:304] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:07:01.936251 16962 raft_consensus.cc:2749] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:01.936941 16956 ts_tablet_manager.cc:1434] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:07:01.937813 16956 tablet_bootstrap.cc:492] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
W20260709 13:07:01.938868 16338 leader_election.cc:336] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.939852 16926 raft_consensus.cc:493] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.940508 16926 raft_consensus.cc:515] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:01.943537 16926 leader_election.cc:290] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:01.947021 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:01.947921 16294 raft_consensus.cc:2468] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
W20260709 13:07:01.950834 16190 leader_election.cc:336] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.956511 16956 tablet_bootstrap.cc:492] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:01.957571 16956 tablet_bootstrap.cc:492] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:01.957337 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c540d71b564c08b9032559d7975cc5" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:01.958280 16956 ts_tablet_manager.cc:1403] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 13:07:01.960453 16956 raft_consensus.cc:359] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:01.960903 16956 raft_consensus.cc:740] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:01.961494 16956 consensus_queue.cc:260] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:07:01.962829 16190 leader_election.cc:336] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:01.963604 16956 ts_tablet_manager.cc:1434] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:07:01.964449 16956 tablet_bootstrap.cc:492] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:01.964395 16188 leader_election.cc:304] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:07:01.965865 16944 raft_consensus.cc:2749] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:01.965798 16337 leader_election.cc:304] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:01.972044 16964 raft_consensus.cc:2804] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:01.972555 16964 raft_consensus.cc:493] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:01.973035 16964 raft_consensus.cc:3060] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.013077 16945 raft_consensus.cc:493] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.013952 16945 raft_consensus.cc:515] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.045426 16944 raft_consensus.cc:493] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.046087 16944 raft_consensus.cc:515] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.048180 16944 leader_election.cc:290] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:07:02.052783 16945 leader_election.cc:290] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:02.049935 16964 raft_consensus.cc:515] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.054642 16962 raft_consensus.cc:493] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.055178 16962 raft_consensus.cc:515] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.056440 16964 leader_election.cc:290] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:07:02.057729 16190 leader_election.cc:336] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.061069 16962 leader_election.cc:290] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:02.062129 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:02.062868 16373 raft_consensus.cc:2468] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
W20260709 13:07:02.064296 16264 leader_election.cc:336] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.069361 16262 leader_election.cc:304] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:02.071714 16373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b5e6c6090448ab831f8afeef43db0d" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:02.076733 16977 raft_consensus.cc:493] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.077329 16977 raft_consensus.cc:515] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:02.086656 16977 leader_election.cc:290] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:02.089375 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c540d71b564c08b9032559d7975cc5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:07:02.090296 16218 raft_consensus.cc:2468] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:07:02.092803 16336 leader_election.cc:304] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:02.094837 16044 catalog_manager.cc:5697] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 1 to 2, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 2 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.103098 16977 raft_consensus.cc:2804] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.103924 16945 raft_consensus.cc:2804] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.068130 16929 raft_consensus.cc:493] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.064756 16962 raft_consensus.cc:493] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.111965 16929 raft_consensus.cc:515] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.112504 16962 raft_consensus.cc:515] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.114449 16962 leader_election.cc:290] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:02.109804 16925 leader_election.cc:290] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:07:02.114678 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:02.111219 16977 raft_consensus.cc:493] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.110913 16945 raft_consensus.cc:493] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.117501 16945 raft_consensus.cc:3060] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.115844 16977 raft_consensus.cc:3060] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.120599 16937 raft_consensus.cc:493] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.121209 16937 raft_consensus.cc:515] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.133347 16956 tablet_bootstrap.cc:492] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.134415 16956 tablet_bootstrap.cc:492] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.135051 16956 ts_tablet_manager.cc:1403] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.171s	user 0.055s	sys 0.054s
I20260709 13:07:02.136382 16771 raft_consensus.cc:493] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
W20260709 13:07:02.139488 16264 leader_election.cc:336] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.140831 16264 leader_election.cc:304] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964
I20260709 13:07:02.142704 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2201023e16254eed9712e5fba24caa38" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:02.143383 16299 raft_consensus.cc:3060] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.137065 16944 raft_consensus.cc:493] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.153218 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:07:02.153265 16189 leader_election.cc:304] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:02.153369 16892 raft_consensus.cc:2749] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.156136 16925 raft_consensus.cc:2749] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:07:02.149863 16337 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 85951.7us
I20260709 13:07:02.137910 16771 raft_consensus.cc:515] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.161651 16944 raft_consensus.cc:515] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.162860 16220 raft_consensus.cc:2468] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
I20260709 13:07:02.169535 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:02.170336 16294 raft_consensus.cc:3060] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.170418 16980 raft_consensus.cc:493] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.171188 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:02.171319 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b5e6c6090448ab831f8afeef43db0d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:02.171836 16297 raft_consensus.cc:2468] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:07:02.172019 16298 raft_consensus.cc:2468] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:07:02.171202 16980 raft_consensus.cc:515] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.173818 16956 raft_consensus.cc:359] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.174435 16944 leader_election.cc:290] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:07:02.174602 16956 raft_consensus.cc:740] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.172924 16771 leader_election.cc:290] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:02.179117 16956 consensus_queue.cc:260] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.182614 16337 leader_election.cc:304] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:02.184794 16956 ts_tablet_manager.cc:1434] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 13:07:02.195185 16956 tablet_bootstrap.cc:492] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:02.160995 16977 raft_consensus.cc:515] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:02.195540 16976 raft_consensus.cc:2804] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.196008 16976 raft_consensus.cc:493] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.196347 16976 raft_consensus.cc:3060] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.188522 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:02.196905 16298 raft_consensus.cc:2468] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 1.
I20260709 13:07:02.187592 16945 raft_consensus.cc:515] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.197280 16977 leader_election.cc:290] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:02.195472 16929 leader_election.cc:290] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:07:02.201761 16338 leader_election.cc:336] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.199067 16337 leader_election.cc:304] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:02.189471 16299 raft_consensus.cc:2468] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 2.
I20260709 13:07:02.190636 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:02.205494 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:02.205674 16297 raft_consensus.cc:2468] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 1.
I20260709 13:07:02.206048 16370 raft_consensus.cc:3060] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.191726 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:07:02.206650 16218 raft_consensus.cc:2468] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 1.
W20260709 13:07:02.191980 16190 leader_election.cc:336] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.192408 16937 leader_election.cc:290] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:07:02.207969 16338 leader_election.cc:336] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
W20260709 13:07:02.208853 16338 leader_election.cc:336] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
W20260709 13:07:02.209518 16338 leader_election.cc:336] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.212080 16945 leader_election.cc:290] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:02.215540 16262 leader_election.cc:304] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:07:02.216575 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42c540d71b564c08b9032559d7975cc5" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:02.217188 16218 raft_consensus.cc:3060] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.217804 16370 raft_consensus.cc:2468] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 2.
I20260709 13:07:02.218637 16962 raft_consensus.cc:2804] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.219766 16262 leader_election.cc:304] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:07:02.219134 16962 raft_consensus.cc:493] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.221266 16962 raft_consensus.cc:3060] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.225771 16189 leader_election.cc:304] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:02.193392 16980 leader_election.cc:290] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:02.226873 16929 raft_consensus.cc:2804] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.227260 16929 raft_consensus.cc:493] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.227582 16929 raft_consensus.cc:3060] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.228799 16294 raft_consensus.cc:2468] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 2.
I20260709 13:07:02.230741 16976 raft_consensus.cc:515] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.232389 16262 leader_election.cc:304] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:02.233008 16976 leader_election.cc:290] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:02.233254 16892 raft_consensus.cc:2804] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.226007 16218 raft_consensus.cc:2468] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 2.
I20260709 13:07:02.220882 16771 raft_consensus.cc:2804] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.233916 16892 raft_consensus.cc:697] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:02.234208 16771 raft_consensus.cc:493] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.234668 16771 raft_consensus.cc:3060] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.235051 16892 consensus_queue.cc:237] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.237170 16929 raft_consensus.cc:515] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.241878 16771 raft_consensus.cc:515] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
W20260709 13:07:02.244009 16337 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 45012.5us
I20260709 13:07:02.244458 16771 leader_election.cc:290] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
W20260709 13:07:02.244906 16264 leader_election.cc:336] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.245600 16337 leader_election.cc:304] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:02.249342 16976 raft_consensus.cc:2804] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:02.249763 16944 raft_consensus.cc:2804] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.249967 16976 raft_consensus.cc:493] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.250715 16944 raft_consensus.cc:697] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 2 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:07:02.251155 16976 raft_consensus.cc:3060] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.256183 16337 leader_election.cc:304] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:02.257282 16944 consensus_queue.cc:237] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.259204 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f16aeeb02645aea6c44a180eac4f87" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:02.259874 16294 raft_consensus.cc:3060] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.261504 16929 leader_election.cc:290] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:02.257275 16980 raft_consensus.cc:2804] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.262902 16980 raft_consensus.cc:697] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:02.264089 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b5e6c6090448ab831f8afeef43db0d" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:02.264026 16980 consensus_queue.cc:237] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.264729 16297 raft_consensus.cc:3060] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.269093 16962 raft_consensus.cc:515] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.269740 16336 leader_election.cc:304] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:02.271067 16981 raft_consensus.cc:2804] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.271677 16981 raft_consensus.cc:697] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:02.272648 16981 consensus_queue.cc:237] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:02.265937 16976 raft_consensus.cc:515] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.283320 16976 leader_election.cc:290] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:02.289105 16047 catalog_manager.cc:5697] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 1 to 2, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:02.290062 16189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 64400.3us
I20260709 13:07:02.290871 16189 leader_election.cc:304] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:02.284989 16297 raft_consensus.cc:2468] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 2.
I20260709 13:07:02.293776 16294 raft_consensus.cc:2468] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 2.
I20260709 13:07:02.294365 16337 leader_election.cc:304] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:02.287935 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181b2ad7f8b649d4be93241715d03572" candidate_uuid: "c8106927ed954e1c800869427b6c7964" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:02.296056 16929 raft_consensus.cc:2804] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.288214 16962 leader_election.cc:290] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:02.295543 16337 leader_election.cc:304] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: 
W20260709 13:07:02.297576 16264 leader_election.cc:336] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.296103 16299 raft_consensus.cc:3060] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.297590 16929 raft_consensus.cc:697] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:02.303037 16976 raft_consensus.cc:2804] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.303557 16976 raft_consensus.cc:697] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
W20260709 13:07:02.304677 16264 leader_election.cc:336] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
W20260709 13:07:02.306146 16338 leader_election.cc:336] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
W20260709 13:07:02.307531 16264 leader_election.cc:336] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
W20260709 13:07:02.307893 16338 leader_election.cc:336] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.304431 16976 consensus_queue.cc:237] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:07:02.312357 16338 leader_election.cc:336] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.308557 16944 raft_consensus.cc:2804] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:07:02.313175 16338 leader_election.cc:336] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
W20260709 13:07:02.314038 16338 leader_election.cc:336] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.313218 16944 raft_consensus.cc:493] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:02.315253 16944 raft_consensus.cc:3060] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.319689 16299 raft_consensus.cc:2468] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8106927ed954e1c800869427b6c7964 in term 2.
I20260709 13:07:02.323827 16929 consensus_queue.cc:237] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.329973 16047 catalog_manager.cc:5697] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 1 to 2, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 2 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.338085 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:02.342725 16337 leader_election.cc:304] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:02.346365 16218 raft_consensus.cc:3060] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:02.325148 16956 tablet_bootstrap.cc:492] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.348156 16956 tablet_bootstrap.cc:492] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.348837 16956 ts_tablet_manager.cc:1403] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.154s	user 0.016s	sys 0.004s
I20260709 13:07:02.338735 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:02.351619 16956 raft_consensus.cc:359] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:02.352411 16956 raft_consensus.cc:740] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.352949 16976 raft_consensus.cc:2804] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.322691 16944 raft_consensus.cc:515] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.354449 16956 consensus_queue.cc:260] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:02.356912 16956 ts_tablet_manager.cc:1434] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:02.357740 16956 tablet_bootstrap.cc:492] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:02.357920 16220 raft_consensus.cc:2393] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 115c2d35eda349178144f8365481e89c in current term 2: Already voted for candidate 3ce6da6d71124256be0679451014f776 in this term.
W20260709 13:07:02.360258 16190 leader_election.cc:336] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:02.353942 16976 raft_consensus.cc:697] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Becoming Leader. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:02.363579 16976 consensus_queue.cc:237] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.369444 16944 leader_election.cc:290] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
W20260709 13:07:02.332212 16264 leader_election.cc:336] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
W20260709 13:07:02.379949 16264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 47762.5us
I20260709 13:07:02.386034 16047 catalog_manager.cc:5697] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 1 to 2, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.396513 16218 raft_consensus.cc:2468] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 2.
I20260709 13:07:02.396473 16047 catalog_manager.cc:5697] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 1 to 2, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.398247 16047 catalog_manager.cc:5697] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 1 to 2, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.399710 16047 catalog_manager.cc:5697] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 1 to 2, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.392838 16370 raft_consensus.cc:1275] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer 115c2d35eda349178144f8365481e89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:02.400279 16262 leader_election.cc:304] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 115c2d35eda349178144f8365481e89c; no voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:02.402060 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:02.402144 16262 leader_election.cc:304] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:02.402900 16294 raft_consensus.cc:2393] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ce6da6d71124256be0679451014f776 in current term 2: Already voted for candidate 115c2d35eda349178144f8365481e89c in this term.
I20260709 13:07:02.403379 16962 raft_consensus.cc:2749] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:07:02.403658 16771 raft_consensus.cc:2804] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:02.404158 16771 raft_consensus.cc:697] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:02.404932 16771 consensus_queue.cc:237] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.413831 16892 consensus_queue.cc:1048] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:02.432226 16189 leader_election.cc:304] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ce6da6d71124256be0679451014f776; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:02.436120 16944 raft_consensus.cc:2749] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:07:02.389377 16264 consensus_peers.cc:597] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:02.444537 16264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 55184.2us
I20260709 13:07:02.446152 16956 tablet_bootstrap.cc:492] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.447192 16956 tablet_bootstrap.cc:492] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.451432 16956 ts_tablet_manager.cc:1403] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.094s	user 0.019s	sys 0.012s
I20260709 13:07:02.453753 16047 catalog_manager.cc:5697] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 reported cstate change: term changed from 1 to 2, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to c8106927ed954e1c800869427b6c7964 (127.7.213.68). New cstate: current_term: 2 leader_uuid: "c8106927ed954e1c800869427b6c7964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:02.481137 16956 raft_consensus.cc:359] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.482074 16956 raft_consensus.cc:740] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.482863 16956 consensus_queue.cc:260] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.485167 16956 ts_tablet_manager.cc:1434] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:07:02.492538 16956 tablet_bootstrap.cc:492] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
W20260709 13:07:02.509140 16264 consensus_peers.cc:597] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:02.528039 16047 catalog_manager.cc:5697] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 1 to 2, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 2 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:02.536231 16956 tablet_bootstrap.cc:492] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.537319 16956 tablet_bootstrap.cc:492] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.538161 16956 ts_tablet_manager.cc:1403] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.046s	user 0.023s	sys 0.000s
I20260709 13:07:02.541080 16956 raft_consensus.cc:359] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.542521 16956 raft_consensus.cc:740] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
W20260709 13:07:02.543215 16338 consensus_peers.cc:597] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:02.543474 16956 consensus_queue.cc:260] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.545753 16956 ts_tablet_manager.cc:1434] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:07:02.546548 16956 tablet_bootstrap.cc:492] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:02.571273 16956 tablet_bootstrap.cc:492] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.572433 16956 tablet_bootstrap.cc:492] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.573196 16956 ts_tablet_manager.cc:1403] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.002s
I20260709 13:07:02.576222 16956 raft_consensus.cc:359] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.577013 16956 raft_consensus.cc:740] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.577731 16956 consensus_queue.cc:260] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.580269 16956 ts_tablet_manager.cc:1434] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:07:02.581033 16956 tablet_bootstrap.cc:492] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
W20260709 13:07:02.606920 16264 consensus_peers.cc:597] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:02.613442 16956 tablet_bootstrap.cc:492] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.614790 16956 tablet_bootstrap.cc:492] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.615620 16956 ts_tablet_manager.cc:1403] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.020s
I20260709 13:07:02.619071 16956 raft_consensus.cc:359] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:02.620046 16956 raft_consensus.cc:740] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.621219 16956 consensus_queue.cc:260] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:02.623919 16956 ts_tablet_manager.cc:1434] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:02.624943 16956 tablet_bootstrap.cc:492] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:02.645550 16956 tablet_bootstrap.cc:492] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.646819 16956 tablet_bootstrap.cc:492] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.647627 16956 ts_tablet_manager.cc:1403] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 13:07:02.650950 16956 raft_consensus.cc:359] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:02.651889 16956 raft_consensus.cc:740] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.664132 16956 consensus_queue.cc:260] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:02.666675 16956 ts_tablet_manager.cc:1434] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:07:02.667429 16956 tablet_bootstrap.cc:492] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
W20260709 13:07:02.670842 16190 consensus_peers.cc:597] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:02.692684 16338 consensus_peers.cc:597] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:02.735494 16218 raft_consensus.cc:1275] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer c8106927ed954e1c800869427b6c7964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:02.736373 16190 consensus_peers.cc:597] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:02.739010 16929 consensus_queue.cc:1048] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:02.757555 16338 consensus_peers.cc:597] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:02.778874 16264 consensus_peers.cc:597] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:02.779965 16956 tablet_bootstrap.cc:492] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.780980 16370 raft_consensus.cc:1275] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer 115c2d35eda349178144f8365481e89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:02.782503 16264 consensus_peers.cc:597] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:02.782688 16771 consensus_queue.cc:1048] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:02.780978 16956 tablet_bootstrap.cc:492] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.783823 16956 ts_tablet_manager.cc:1403] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.117s	user 0.021s	sys 0.004s
I20260709 13:07:02.786736 16956 raft_consensus.cc:359] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.787477 16956 raft_consensus.cc:740] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.790232 16956 consensus_queue.cc:260] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.804078 16299 raft_consensus.cc:1275] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer 3ce6da6d71124256be0679451014f776: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:02.804777 16956 ts_tablet_manager.cc:1434] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:07:02.806365 16956 tablet_bootstrap.cc:492] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:02.806830 16925 consensus_queue.cc:1048] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:02.818225 16190 consensus_peers.cc:597] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:02.822100 16190 consensus_peers.cc:597] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:02.836050 16264 consensus_peers.cc:597] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:02.848878 16956 tablet_bootstrap.cc:492] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.849905 16956 tablet_bootstrap.cc:492] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.850744 16956 ts_tablet_manager.cc:1403] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.000s
W20260709 13:07:02.853993 16264 consensus_peers.cc:597] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:02.853935 16956 raft_consensus.cc:359] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
W20260709 13:07:02.854486 16264 consensus_peers.cc:597] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:02.854619 16956 raft_consensus.cc:740] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.855711 16956 consensus_queue.cc:260] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:02.857678 16956 ts_tablet_manager.cc:1434] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:07:02.858461 16956 tablet_bootstrap.cc:492] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
W20260709 13:07:02.861538 16190 consensus_peers.cc:597] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:02.864526 16218 raft_consensus.cc:1275] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer 115c2d35eda349178144f8365481e89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:02.867069 16962 consensus_queue.cc:1048] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:02.871377 16299 raft_consensus.cc:1275] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer c8106927ed954e1c800869427b6c7964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:02.872834 16980 consensus_queue.cc:1048] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:02.889696 16956 tablet_bootstrap.cc:492] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:02.891088 16956 tablet_bootstrap.cc:492] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:02.891952 16956 ts_tablet_manager.cc:1403] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.012s
I20260709 13:07:02.896718 16956 raft_consensus.cc:359] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.897676 16956 raft_consensus.cc:740] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:02.898705 16956 consensus_queue.cc:260] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:02.905277 16956 ts_tablet_manager.cc:1434] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:07:02.906045 16956 tablet_bootstrap.cc:492] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:02.908474 16294 raft_consensus.cc:1275] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer c8106927ed954e1c800869427b6c7964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:02.911216 16937 consensus_queue.cc:1048] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:02.946587 16190 consensus_peers.cc:597] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:02.971462 16338 consensus_peers.cc:597] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:02.972280 16338 consensus_peers.cc:597] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:02.984843 16190 consensus_peers.cc:597] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:02.993973 16264 consensus_peers.cc:597] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:02.997963 16264 consensus_peers.cc:597] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:02.998987 16956 tablet_bootstrap.cc:492] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:07:03.000265 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 48126.5us
W20260709 13:07:03.000762 16338 consensus_peers.cc:597] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.003721 16294 raft_consensus.cc:1275] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer c8106927ed954e1c800869427b6c7964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:03.005486 16338 consensus_peers.cc:597] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:03.006017 16338 consensus_peers.cc:597] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:03.007284 16299 raft_consensus.cc:1275] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer c8106927ed954e1c800869427b6c7964: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:07:03.009212 16190 consensus_peers.cc:597] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.000793 16956 tablet_bootstrap.cc:492] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:03.019095 16956 ts_tablet_manager.cc:1403] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.113s	user 0.020s	sys 0.033s
I20260709 13:07:03.021840 16956 raft_consensus.cc:359] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.022528 16956 raft_consensus.cc:740] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.023223 16956 consensus_queue.cc:260] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.025342 16956 ts_tablet_manager.cc:1434] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:07:03.026118 16956 tablet_bootstrap.cc:492] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:03.027721 16976 consensus_queue.cc:1048] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:03.031801 16976 consensus_queue.cc:1048] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:07:03.039014 16190 consensus_peers.cc:597] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.040174 16338 consensus_peers.cc:597] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.046945 16190 consensus_peers.cc:597] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.047711 16958 raft_consensus.cc:493] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:03.047518 16338 consensus_peers.cc:597] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.048219 16958 raft_consensus.cc:515] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:03.053113 16958 leader_election.cc:290] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
W20260709 13:07:03.067055 16264 consensus_peers.cc:597] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.106765 16190 consensus_peers.cc:597] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.125079 16190 consensus_peers.cc:597] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.125837 16190 consensus_peers.cc:597] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.176111 16190 consensus_peers.cc:597] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.178936 16338 consensus_peers.cc:597] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.180284 16958 raft_consensus.cc:493] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.184227 16958 raft_consensus.cc:515] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:03.187460 16958 leader_election.cc:290] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:03.188284 16958 raft_consensus.cc:493] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.188989 16958 raft_consensus.cc:515] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:03.207666 16958 leader_election.cc:290] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:03.223956 16956 tablet_bootstrap.cc:492] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.227028 16956 tablet_bootstrap.cc:492] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:03.227667 16956 ts_tablet_manager.cc:1403] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.202s	user 0.055s	sys 0.078s
I20260709 13:07:03.230139 16956 raft_consensus.cc:359] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:03.234870 16956 raft_consensus.cc:740] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.235558 16956 consensus_queue.cc:260] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:03.241374 16956 ts_tablet_manager.cc:1434] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:07:03.242273 16956 tablet_bootstrap.cc:492] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:03.246272 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62345643f7684ec08f60f740de27cead" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:07:03.248857 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047f65a57d24cafbf3f41f5b235556e" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:07:03.249366 16264 consensus_peers.cc:597] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:03.249966 16264 consensus_peers.cc:597] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.249998 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
W20260709 13:07:03.259917 16264 consensus_peers.cc:597] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.261783 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:03.264209 16950 leader_election.cc:304] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:07:03.265341 16958 raft_consensus.cc:2749] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.266055 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1047f65a57d24cafbf3f41f5b235556e" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:03.268148 16950 leader_election.cc:304] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:07:03.269155 16958 raft_consensus.cc:2749] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.277700 16958 raft_consensus.cc:493] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.278168 16958 raft_consensus.cc:515] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:03.281177 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:03.284082 16958 leader_election.cc:290] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:03.287038 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a042ac05958049c38414fa65d0c39414" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:07:03.289515 16950 leader_election.cc:304] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:07:03.292963 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62345643f7684ec08f60f740de27cead" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:03.296849 16951 leader_election.cc:304] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776
I20260709 13:07:03.298084 17071 raft_consensus.cc:2749] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.298457 16958 raft_consensus.cc:2749] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.299877 16956 tablet_bootstrap.cc:492] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.301121 16956 tablet_bootstrap.cc:492] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:03.302006 16956 ts_tablet_manager.cc:1403] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.060s	user 0.014s	sys 0.020s
W20260709 13:07:03.310873 16264 consensus_peers.cc:597] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.344038  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:40593
I20260709 13:07:03.434902 16956 raft_consensus.cc:359] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.435703 16956 raft_consensus.cc:740] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.436638 16956 consensus_queue.cc:260] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.439821 16956 ts_tablet_manager.cc:1434] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.137s	user 0.015s	sys 0.027s
I20260709 13:07:03.440415 17080 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:40593 every 8 connection(s)
I20260709 13:07:03.441008 16956 tablet_bootstrap.cc:492] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:03.473687 16958 raft_consensus.cc:493] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.474248 16958 raft_consensus.cc:515] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.476282 16958 leader_election.cc:290] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:03.477916 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:03.479910 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:03.481992 16951 leader_election.cc:304] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:07:03.482862 16958 raft_consensus.cc:2749] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.493980 16956 tablet_bootstrap.cc:492] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.498823 16956 tablet_bootstrap.cc:492] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:03.498863 17081 heartbeater.cc:344] Connected to a master server at 127.7.213.126:46449
I20260709 13:07:03.504283 17081 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:03.505188 17081 heartbeater.cc:507] Master 127.7.213.126:46449 requested a full tablet report, sending...
I20260709 13:07:03.509158 16956 ts_tablet_manager.cc:1403] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.068s	user 0.018s	sys 0.000s
I20260709 13:07:03.511708 16956 raft_consensus.cc:359] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.512876 16956 raft_consensus.cc:740] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.514073 16956 consensus_queue.cc:260] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.516834 16956 ts_tablet_manager.cc:1434] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:07:03.517897 16956 tablet_bootstrap.cc:492] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:03.525668 17031 raft_consensus.cc:3060] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:03.535384 16264 consensus_peers.cc:597] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.536931 17033 raft_consensus.cc:3060] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:03.537775 16338 consensus_peers.cc:597] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:07:03.560330 17089 raft_consensus.cc:493] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.560946 17089 raft_consensus.cc:515] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
W20260709 13:07:03.562716 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 51171.3us
I20260709 13:07:03.568563 16047 ts_manager.cc:194] Re-registered known tserver with Master: aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
I20260709 13:07:03.576018 16958 raft_consensus.cc:493] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:03.578872 16958 raft_consensus.cc:515] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.581182 16958 leader_election.cc:290] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:03.582929 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:03.583596 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ce6da6d71124256be0679451014f776" is_pre_election: true
I20260709 13:07:03.584334 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:03.585597 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abb840ca52d04206a82103fb4eb79ccf" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c8106927ed954e1c800869427b6c7964" is_pre_election: true
I20260709 13:07:03.587210 16950 leader_election.cc:304] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 115c2d35eda349178144f8365481e89c, c8106927ed954e1c800869427b6c7964
I20260709 13:07:03.588073 17089 leader_election.cc:290] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:03.588408 16950 leader_election.cc:304] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d; no voters: 3ce6da6d71124256be0679451014f776, c8106927ed954e1c800869427b6c7964
I20260709 13:07:03.588825 17089 raft_consensus.cc:2749] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.589751 17089 raft_consensus.cc:2749] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:03.595395 16956 tablet_bootstrap.cc:492] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.596467 16956 tablet_bootstrap.cc:492] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:03.597137 16956 ts_tablet_manager.cc:1403] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.079s	user 0.027s	sys 0.039s
I20260709 13:07:03.600147 16956 raft_consensus.cc:359] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:03.600844 16956 raft_consensus.cc:740] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.601504 16956 consensus_queue.cc:260] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:03.603833 16956 ts_tablet_manager.cc:1434] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:07:03.604612 16956 tablet_bootstrap.cc:492] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:03.634449 16956 tablet_bootstrap.cc:492] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.635439 16956 tablet_bootstrap.cc:492] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:03.636142 16956 ts_tablet_manager.cc:1403] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.032s	user 0.021s	sys 0.008s
I20260709 13:07:03.638854 16956 raft_consensus.cc:359] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:03.639639 16956 raft_consensus.cc:740] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.640424 16956 consensus_queue.cc:260] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:03.642863 16956 ts_tablet_manager.cc:1434] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:03.643874 16956 tablet_bootstrap.cc:492] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:03.647881 16047 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51964
I20260709 13:07:03.655367 17081 heartbeater.cc:499] Master 127.7.213.126:46449 was elected leader, sending a full tablet report...
I20260709 13:07:03.681306 16956 tablet_bootstrap.cc:492] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.682456 16956 tablet_bootstrap.cc:492] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:03.683192 16956 ts_tablet_manager.cc:1403] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.004s
I20260709 13:07:03.685797 16956 raft_consensus.cc:359] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.686470 16956 raft_consensus.cc:740] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.687160 16956 consensus_queue.cc:260] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.689204 16956 ts_tablet_manager.cc:1434] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:07:03.689988 16956 tablet_bootstrap.cc:492] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:03.705915 17031 raft_consensus.cc:3060] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:03.768288 16956 tablet_bootstrap.cc:492] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:03.769277 16956 tablet_bootstrap.cc:492] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:03.769968 16956 ts_tablet_manager.cc:1403] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.080s	user 0.010s	sys 0.031s
I20260709 13:07:03.774925 17026 raft_consensus.cc:3060] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:03.775203 16956 raft_consensus.cc:359] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.782196 16956 raft_consensus.cc:740] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:03.783010 16956 consensus_queue.cc:260] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:03.785236 16956 ts_tablet_manager.cc:1434] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:07:03.786170 16956 tablet_bootstrap.cc:492] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:03.813171 17033 raft_consensus.cc:3060] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:03.875351 17031 raft_consensus.cc:3060] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.000560 17028 raft_consensus.cc:3060] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:04.071341 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 85606.1us
W20260709 13:07:04.081071 16337 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 47505.6us
W20260709 13:07:04.134759 16337 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 47547.5us
I20260709 13:07:04.210106 16944 raft_consensus.cc:493] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:04.210721 16264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 137717us
I20260709 13:07:04.210811 16944 raft_consensus.cc:515] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:04.214645 16944 leader_election.cc:290] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:07:04.226306 17032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d" is_pre_election: true
W20260709 13:07:04.232686 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:43495, user_credentials={real_user=slave}} blocked reactor thread for 58127.2us
W20260709 13:07:04.247644 16338 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 54836.9us
W20260709 13:07:04.254228 16189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 54227.2us
W20260709 13:07:04.258075 16264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 46274.5us
W20260709 13:07:04.331069 16262 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 44403.2us
W20260709 13:07:04.335466 16956 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.141s	sys 0.273s
W20260709 13:07:04.336010 16956 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.141s	sys 0.273s
W20260709 13:07:04.339020 16190 leader_election.cc:343] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:04.349615 16956 tablet_bootstrap.cc:492] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.389755 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c" is_pre_election: true
I20260709 13:07:04.391423 16956 tablet_bootstrap.cc:492] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:04.392550 16956 ts_tablet_manager.cc:1403] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.607s	user 0.158s	sys 0.273s
I20260709 13:07:04.390514 16299 raft_consensus.cc:2468] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 2.
I20260709 13:07:04.399322 16189 leader_election.cc:304] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
W20260709 13:07:04.401446 16264 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:40593, user_credentials={real_user=slave}} blocked reactor thread for 49063.7us
I20260709 13:07:04.405469 17114 raft_consensus.cc:2804] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:07:04.406081 17114 raft_consensus.cc:493] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:04.412748 17114 raft_consensus.cc:3060] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.440579 16956 raft_consensus.cc:359] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:04.441879 16956 raft_consensus.cc:740] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.442948 16956 consensus_queue.cc:260] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:04.445693 16956 ts_tablet_manager.cc:1434] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:07:04.450829 17114 raft_consensus.cc:515] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:04.463588 17114 leader_election.cc:290] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 3 election: Requested vote from peers aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:07:04.471784 16956 tablet_bootstrap.cc:492] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:04.475651 17030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
W20260709 13:07:04.477833 16190 leader_election.cc:343] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:04.480036 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c66825803ddc46e8849c7718d3cb56ea" candidate_uuid: "3ce6da6d71124256be0679451014f776" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:04.480679 16299 raft_consensus.cc:3060] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:04.490334 16299 raft_consensus.cc:2468] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ce6da6d71124256be0679451014f776 in term 3.
I20260709 13:07:04.498065 16189 leader_election.cc:304] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:04.498963 17115 raft_consensus.cc:2804] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:04.499446 17115 raft_consensus.cc:697] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 3 LEADER]: Becoming Leader. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:07:04.500293 17115 consensus_queue.cc:237] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:04.537103 16956 tablet_bootstrap.cc:492] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.538419 16956 tablet_bootstrap.cc:492] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:04.539264 16956 ts_tablet_manager.cc:1403] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.068s	user 0.017s	sys 0.000s
I20260709 13:07:04.541956 16956 raft_consensus.cc:359] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:04.542744 16956 raft_consensus.cc:740] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.543610 16956 consensus_queue.cc:260] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:04.545936 16047 catalog_manager.cc:5697] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 reported cstate change: term changed from 1 to 3, leader changed from aa8eb07867e7420b80a7e615d570509d (127.7.213.65) to 3ce6da6d71124256be0679451014f776 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "3ce6da6d71124256be0679451014f776" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:04.558454 16956 ts_tablet_manager.cc:1434] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:07:04.566344 16956 tablet_bootstrap.cc:492] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:04.610179 16956 tablet_bootstrap.cc:492] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.616137 16956 tablet_bootstrap.cc:492] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:04.616963 16956 ts_tablet_manager.cc:1403] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.005s
I20260709 13:07:04.620046 16956 raft_consensus.cc:359] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:04.621369 16956 raft_consensus.cc:740] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.622958 16956 consensus_queue.cc:260] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:04.630882 16956 ts_tablet_manager.cc:1434] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:07:04.631999 16956 tablet_bootstrap.cc:492] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:04.658905 17031 raft_consensus.cc:3060] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.708086 16956 tablet_bootstrap.cc:492] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.714272 16956 tablet_bootstrap.cc:492] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:04.715458 16956 ts_tablet_manager.cc:1403] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.084s	user 0.020s	sys 0.005s
I20260709 13:07:04.718506 16956 raft_consensus.cc:359] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:04.721470 16956 raft_consensus.cc:740] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.722321 16956 consensus_queue.cc:260] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:04.726135 16956 ts_tablet_manager.cc:1434] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:07:04.729297 16956 tablet_bootstrap.cc:492] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:04.752983 16956 tablet_bootstrap.cc:492] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.753813 16956 tablet_bootstrap.cc:492] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:04.754462 16956 ts_tablet_manager.cc:1403] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.009s
I20260709 13:07:04.758157 16956 raft_consensus.cc:359] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:04.758955 16956 raft_consensus.cc:740] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.759773 16956 consensus_queue.cc:260] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:04.762113 16956 ts_tablet_manager.cc:1434] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:04.763005 16956 tablet_bootstrap.cc:492] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:04.831766 16956 tablet_bootstrap.cc:492] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.832896 16956 tablet_bootstrap.cc:492] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:04.833678 16956 ts_tablet_manager.cc:1403] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.071s	user 0.027s	sys 0.033s
I20260709 13:07:04.839512 16956 raft_consensus.cc:359] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:04.841746 16956 raft_consensus.cc:740] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.842657 16956 consensus_queue.cc:260] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:04.875100 16956 ts_tablet_manager.cc:1434] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.041s	user 0.011s	sys 0.012s
I20260709 13:07:04.882695 17031 raft_consensus.cc:3060] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:04.886586 16956 tablet_bootstrap.cc:492] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:04.917435 16299 raft_consensus.cc:1275] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Refusing update from remote peer 3ce6da6d71124256be0679451014f776: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:04.920543 17105 consensus_queue.cc:1048] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:04.952481 16190 consensus_peers.cc:597] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:04.972488 16956 tablet_bootstrap.cc:492] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:04.978755 16956 tablet_bootstrap.cc:492] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:04.981326 16956 ts_tablet_manager.cc:1403] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.095s	user 0.021s	sys 0.004s
I20260709 13:07:04.986686 16956 raft_consensus.cc:359] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:04.987888 16956 raft_consensus.cc:740] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:04.988583 16956 consensus_queue.cc:260] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:04.991056 16956 ts_tablet_manager.cc:1434] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 13:07:04.993063 16956 tablet_bootstrap.cc:492] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.030200 16956 tablet_bootstrap.cc:492] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.030939 16956 tablet_bootstrap.cc:492] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.031463 16956 ts_tablet_manager.cc:1403] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.007s
I20260709 13:07:05.033383 16956 raft_consensus.cc:359] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:05.033929 16956 raft_consensus.cc:740] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.034554 16956 consensus_queue.cc:260] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:05.036463 16956 ts_tablet_manager.cc:1434] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:07:05.037694 16956 tablet_bootstrap.cc:492] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.057735 16956 tablet_bootstrap.cc:492] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.058768 16956 tablet_bootstrap.cc:492] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.059458 16956 ts_tablet_manager.cc:1403] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 13:07:05.062005 16956 raft_consensus.cc:359] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:05.062654 16956 raft_consensus.cc:740] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.063313 16956 consensus_queue.cc:260] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:05.065222 16956 ts_tablet_manager.cc:1434] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:05.066301 16956 tablet_bootstrap.cc:492] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.080013 16956 tablet_bootstrap.cc:492] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.080964 16956 tablet_bootstrap.cc:492] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.081653 16956 ts_tablet_manager.cc:1403] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 13:07:05.083878 16956 raft_consensus.cc:359] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:05.084581 16956 raft_consensus.cc:740] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.085319 16956 consensus_queue.cc:260] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:05.089761 16956 ts_tablet_manager.cc:1434] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.004s
I20260709 13:07:05.090621 16956 tablet_bootstrap.cc:492] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.106564 16956 tablet_bootstrap.cc:492] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.107589 16956 tablet_bootstrap.cc:492] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.108328 16956 ts_tablet_manager.cc:1403] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:07:05.110579 16956 raft_consensus.cc:359] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:05.111213 16956 raft_consensus.cc:740] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.111855 16956 consensus_queue.cc:260] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:05.113514 16956 ts_tablet_manager.cc:1434] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:05.117154 16956 tablet_bootstrap.cc:492] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.149572 16956 tablet_bootstrap.cc:492] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.150691 16956 tablet_bootstrap.cc:492] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.151348 16956 ts_tablet_manager.cc:1403] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.016s
I20260709 13:07:05.153741 16956 raft_consensus.cc:359] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.154417 16956 raft_consensus.cc:740] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.155289 16956 consensus_queue.cc:260] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.160507 16956 ts_tablet_manager.cc:1434] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:07:05.161449 16956 tablet_bootstrap.cc:492] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.184481 16956 tablet_bootstrap.cc:492] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.185401 16956 tablet_bootstrap.cc:492] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.185977 16956 ts_tablet_manager.cc:1403] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.003s
I20260709 13:07:05.187914 16956 raft_consensus.cc:359] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.188593 16956 raft_consensus.cc:740] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.189262 16956 consensus_queue.cc:260] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.191293 16956 ts_tablet_manager.cc:1434] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:07:05.192101 16956 tablet_bootstrap.cc:492] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.208165 16956 tablet_bootstrap.cc:492] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.208880 16956 tablet_bootstrap.cc:492] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.209427 16956 ts_tablet_manager.cc:1403] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 13:07:05.211856 16956 raft_consensus.cc:359] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.212554 16956 raft_consensus.cc:740] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.213259 16956 consensus_queue.cc:260] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.215374 16956 ts_tablet_manager.cc:1434] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:07:05.216238 16956 tablet_bootstrap.cc:492] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.240157 16956 tablet_bootstrap.cc:492] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.241179 16956 tablet_bootstrap.cc:492] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.242122 16956 ts_tablet_manager.cc:1403] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.003s
I20260709 13:07:05.245159 16956 raft_consensus.cc:359] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.246021 16956 raft_consensus.cc:740] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.246740 16956 consensus_queue.cc:260] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.248942 16956 ts_tablet_manager.cc:1434] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:05.250399 16956 tablet_bootstrap.cc:492] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
W20260709 13:07:05.263527 16190 consensus_peers.cc:597] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:05.320221 16956 tablet_bootstrap.cc:492] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.321347 16956 tablet_bootstrap.cc:492] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.322083 16956 ts_tablet_manager.cc:1403] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.072s	user 0.036s	sys 0.028s
I20260709 13:07:05.325011 16956 raft_consensus.cc:359] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:05.325776 16956 raft_consensus.cc:740] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.326550 16956 consensus_queue.cc:260] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:05.328652 16956 ts_tablet_manager.cc:1434] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:05.332743 16956 tablet_bootstrap.cc:492] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.366171 16956 tablet_bootstrap.cc:492] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.367113 16956 tablet_bootstrap.cc:492] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.367790 16956 ts_tablet_manager.cc:1403] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.011s
I20260709 13:07:05.371335 16956 raft_consensus.cc:359] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:05.372092 16956 raft_consensus.cc:740] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.372779 16956 consensus_queue.cc:260] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:05.374409 16956 ts_tablet_manager.cc:1434] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:05.375212 16956 tablet_bootstrap.cc:492] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.408941 16956 tablet_bootstrap.cc:492] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.410003 16956 tablet_bootstrap.cc:492] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.410740 16956 ts_tablet_manager.cc:1403] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.011s
I20260709 13:07:05.413682 16956 raft_consensus.cc:359] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.414397 16956 raft_consensus.cc:740] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.415158 16956 consensus_queue.cc:260] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:05.428910 16956 ts_tablet_manager.cc:1434] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:07:05.466329 16956 tablet_bootstrap.cc:492] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: Bootstrap starting.
I20260709 13:07:05.500520 16956 tablet_bootstrap.cc:492] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:05.501673 16956 tablet_bootstrap.cc:492] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: Bootstrap complete.
I20260709 13:07:05.502490 16956 ts_tablet_manager.cc:1403] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.000s
I20260709 13:07:05.505451 16956 raft_consensus.cc:359] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:05.506556 16956 raft_consensus.cc:740] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Initialized, Role: FOLLOWER
I20260709 13:07:05.507344 16956 consensus_queue.cc:260] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:05.509457 16956 ts_tablet_manager.cc:1434] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
W20260709 13:07:05.515036 16188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 140890us
I20260709 13:07:05.797355 17030 raft_consensus.cc:3060] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:07:08.867329 16189 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:36993, user_credentials={real_user=slave}} blocked reactor thread for 164579us
W20260709 13:07:08.891237 16188 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.68:36791, user_credentials={real_user=slave}} blocked reactor thread for 49267.7us
W20260709 13:07:09.888880 16320 debug-util.cc:398] Leaking SignalData structure 0x7b08002baa00 after lost signal to thread 8024
W20260709 13:07:09.890028 16320 debug-util.cc:398] Leaking SignalData structure 0x7b08000dd7a0 after lost signal to thread 16082
W20260709 13:07:09.891265 16320 debug-util.cc:398] Leaking SignalData structure 0x7b08002777a0 after lost signal to thread 16249
W20260709 13:07:09.892369 16320 debug-util.cc:398] Leaking SignalData structure 0x7b08001dd940 after lost signal to thread 16323
W20260709 13:07:09.893463 16320 debug-util.cc:398] Leaking SignalData structure 0x7b0800267ec0 after lost signal to thread 16397
W20260709 13:07:09.894534 16320 debug-util.cc:398] Leaking SignalData structure 0x7b0800036540 after lost signal to thread 17080
I20260709 13:07:13.471184  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:13.533161 16219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba"
dest_uuid: "3ce6da6d71124256be0679451014f776"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:55110
I20260709 13:07:13.533913 16219 raft_consensus.cc:606] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:13.547479 16298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1"
dest_uuid: "115c2d35eda349178144f8365481e89c"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:44392
I20260709 13:07:13.548020 16298 raft_consensus.cc:606] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:13.556152 17177 raft_consensus.cc:993] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.556759 17178 raft_consensus.cc:1081] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.558761 17033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45066
I20260709 13:07:13.559443 17033 raft_consensus.cc:493] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.559921 17033 raft_consensus.cc:3060] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.563971 16372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5"
dest_uuid: "c8106927ed954e1c800869427b6c7964"
mode: GRACEFUL
new_leader_uuid: "115c2d35eda349178144f8365481e89c"
 from {username='slave'} at 127.0.0.1:40076
I20260709 13:07:13.564616 16372 raft_consensus.cc:606] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Received request to transfer leadership to TS 115c2d35eda349178144f8365481e89c
I20260709 13:07:13.567873 16298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58b7aa0c7f8b4bf4a2e4881b23351045"
dest_uuid: "115c2d35eda349178144f8365481e89c"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:44392
I20260709 13:07:13.568329 16298 raft_consensus.cc:606] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:13.569175 17033 raft_consensus.cc:515] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:13.571542 17033 leader_election.cc:290] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:13.572763 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:13.573016 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18645784203b4ce1bbe3302513d910c1" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:13.573763 16372 raft_consensus.cc:3060] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.573732 16299 raft_consensus.cc:3055] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.574337 16299 raft_consensus.cc:740] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:13.575431 16299 consensus_queue.cc:260] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:13.576895 16299 raft_consensus.cc:3060] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.570986 16298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa"
dest_uuid: "115c2d35eda349178144f8365481e89c"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:44392
I20260709 13:07:13.579798 16298 raft_consensus.cc:606] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:13.587292 16299 raft_consensus.cc:2468] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:13.587816 16298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978"
dest_uuid: "115c2d35eda349178144f8365481e89c"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:44392
I20260709 13:07:13.588711 16298 raft_consensus.cc:606] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:13.588857 16951 leader_election.cc:304] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:07:13.587873 16372 raft_consensus.cc:2468] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:13.595688 16299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84"
dest_uuid: "115c2d35eda349178144f8365481e89c"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:44392
I20260709 13:07:13.596269 17186 raft_consensus.cc:2804] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.596465 16299 raft_consensus.cc:606] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:13.598260 17186 raft_consensus.cc:697] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:13.598284  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:07:13.599092 17186 consensus_queue.cc:237] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:13.599962  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:13.614390 16045 catalog_manager.cc:5697] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from 115c2d35eda349178144f8365481e89c (127.7.213.67) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:13.684607 17178 raft_consensus.cc:993] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.685105 17177 raft_consensus.cc:1081] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.686625 17033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58b7aa0c7f8b4bf4a2e4881b23351045"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45066
I20260709 13:07:13.687268 17033 raft_consensus.cc:493] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.687654 17033 raft_consensus.cc:3060] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.694490 17033 raft_consensus.cc:515] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:13.696205 17033 leader_election.cc:290] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:07:13.697175 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b7aa0c7f8b4bf4a2e4881b23351045" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:13.697299 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58b7aa0c7f8b4bf4a2e4881b23351045" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:13.698093 16298 raft_consensus.cc:3055] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.698153 16225 raft_consensus.cc:3060] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.698549 16298 raft_consensus.cc:740] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:13.699571 16298 consensus_queue.cc:260] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:13.701206 16298 raft_consensus.cc:3060] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.706279 16225 raft_consensus.cc:2468] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:13.709292 16298 raft_consensus.cc:2468] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:13.710122 16950 leader_election.cc:304] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:07:13.710666 17178 raft_consensus.cc:993] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.711340 17177 raft_consensus.cc:1081] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.711376 17186 raft_consensus.cc:2804] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.712085 17186 raft_consensus.cc:697] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:13.713207 17033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45066
I20260709 13:07:13.713204 17186 consensus_queue.cc:237] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:13.714098 17033 raft_consensus.cc:493] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.714478 17033 raft_consensus.cc:3060] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.722009 17033 raft_consensus.cc:515] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:13.724054 17033 leader_election.cc:290] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:07:13.725957 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:13.726102 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829bf55d32c344af8ae546eb15903faa" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:13.726577 16298 raft_consensus.cc:3055] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.726739 16225 raft_consensus.cc:3060] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.726986 16298 raft_consensus.cc:740] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:13.726107 16045 catalog_manager.cc:5697] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from 115c2d35eda349178144f8365481e89c (127.7.213.67) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.728020 16298 consensus_queue.cc:260] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:13.730367 16298 raft_consensus.cc:3060] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.735651 16225 raft_consensus.cc:2468] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:13.736864 16950 leader_election.cc:304] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:07:13.737675 17186 raft_consensus.cc:2804] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.738193 17186 raft_consensus.cc:697] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:13.739219 17186 consensus_queue.cc:237] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:13.741194 16298 raft_consensus.cc:2468] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:13.749405 16045 catalog_manager.cc:5697] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from 115c2d35eda349178144f8365481e89c (127.7.213.67) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.859156 17168 raft_consensus.cc:993] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964: : Instructing follower 115c2d35eda349178144f8365481e89c to start an election
I20260709 13:07:13.859699 17180 raft_consensus.cc:1081] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Signalling peer 115c2d35eda349178144f8365481e89c to start an election
I20260709 13:07:13.861306 16298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5"
dest_uuid: "115c2d35eda349178144f8365481e89c"
 from {username='slave'} at 127.0.0.1:37816
I20260709 13:07:13.861789 16298 raft_consensus.cc:493] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.862141 16298 raft_consensus.cc:3060] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.866904 16298 raft_consensus.cc:515] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:07:13.868796 16298 leader_election.cc:290] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:13.869705 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:13.869930 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31991d68fa8642189423a50030c4e7d5" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:13.870303 16225 raft_consensus.cc:3060] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.870529 16372 raft_consensus.cc:3055] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.870882 16372 raft_consensus.cc:740] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:13.871788 16372 consensus_queue.cc:260] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:07:13.872926 16372 raft_consensus.cc:3060] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.876369 16225 raft_consensus.cc:2468] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 2.
I20260709 13:07:13.877629 16372 raft_consensus.cc:2468] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 2.
I20260709 13:07:13.877847 16262 leader_election.cc:304] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:07:13.881251 17177 raft_consensus.cc:2804] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.881865 17177 raft_consensus.cc:697] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:13.882630 17177 consensus_queue.cc:237] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:07:13.893016 16045 catalog_manager.cc:5697] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 1 to 2, leader changed from c8106927ed954e1c800869427b6c7964 (127.7.213.68) to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 2 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.906813 17172 raft_consensus.cc:993] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.907369 17176 raft_consensus.cc:1081] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.908720 17033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45082
I20260709 13:07:13.909231 17033 raft_consensus.cc:493] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.909521 17033 raft_consensus.cc:3060] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.915273 17033 raft_consensus.cc:515] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:13.916890 17033 leader_election.cc:290] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:13.917515 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:13.917872 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09715ac5d7d741e3aabf462dcab633ba" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:13.918213 16372 raft_consensus.cc:3060] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.918476 16225 raft_consensus.cc:3055] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.918816 16225 raft_consensus.cc:740] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:07:13.919440 16225 consensus_queue.cc:260] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:13.920450 16225 raft_consensus.cc:3060] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.923226 16372 raft_consensus.cc:2468] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:13.924311 16950 leader_election.cc:304] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:13.925128 17186 raft_consensus.cc:2804] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.925585 17186 raft_consensus.cc:697] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:13.926225 16225 raft_consensus.cc:2468] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:13.926434 17186 consensus_queue.cc:237] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:13.935855 16045 catalog_manager.cc:5697] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from 3ce6da6d71124256be0679451014f776 (127.7.213.66) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.982023 17178 raft_consensus.cc:993] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.982550 17177 raft_consensus.cc:1081] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:13.984269 17032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45066
I20260709 13:07:13.984738 17032 raft_consensus.cc:493] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.985013 17032 raft_consensus.cc:3060] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:13.990685 17032 raft_consensus.cc:515] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:13.993600 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:13.993911 17032 leader_election.cc:290] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 3 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:13.994266 16298 raft_consensus.cc:3055] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:13.994663 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b5ad93fb43d4467ac4dc1f70ccb0978" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:13.994690 16298 raft_consensus.cc:740] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:13.995393 16372 raft_consensus.cc:3060] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:13.996121 16298 consensus_queue.cc:260] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:13.997757 16298 raft_consensus.cc:3060] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:13.998011 16370 raft_consensus.cc:1275] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.001691 17186 consensus_queue.cc:1048] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.003113 16372 raft_consensus.cc:2468] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 3.
I20260709 13:07:14.004845 16950 leader_election.cc:304] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:14.005806 17186 raft_consensus.cc:2804] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:14.009979 16298 raft_consensus.cc:2468] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 3.
I20260709 13:07:14.006374 17186 raft_consensus.cc:697] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 3 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:14.013648 17186 consensus_queue.cc:237] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:14.044394 16045 catalog_manager.cc:5697] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 2 to 3, leader changed from 115c2d35eda349178144f8365481e89c (127.7.213.67) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 3 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:14.044529 17185 raft_consensus.cc:993] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:14.050613 17185 raft_consensus.cc:1081] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:14.050871 16298 raft_consensus.cc:1275] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.052425 17032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45066
I20260709 13:07:14.053031 17032 raft_consensus.cc:493] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.053485 17032 raft_consensus.cc:3060] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.053267 17198 consensus_queue.cc:1048] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:14.065699 17032 raft_consensus.cc:515] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:14.069020 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:14.069671 16225 raft_consensus.cc:3060] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.074489 17032 leader_election.cc:290] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 3 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), 115c2d35eda349178144f8365481e89c (127.7.213.67:36993)
I20260709 13:07:14.074908 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f5d69137b6e468bb06eb4212b198f84" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:14.075527 16299 raft_consensus.cc:3055] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:14.075839 16299 raft_consensus.cc:740] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:14.076706 16299 consensus_queue.cc:260] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:14.077863 16299 raft_consensus.cc:3060] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:14.086656 16299 raft_consensus.cc:2468] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 3.
I20260709 13:07:14.087910 16951 leader_election.cc:304] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:07:14.089184 17198 raft_consensus.cc:2804] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:14.090294 17198 raft_consensus.cc:697] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 3 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:14.091338 16225 raft_consensus.cc:2468] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 3.
I20260709 13:07:14.091637 17198 consensus_queue.cc:237] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } }
I20260709 13:07:14.103019 16045 catalog_manager.cc:5697] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 2 to 3, leader changed from 115c2d35eda349178144f8365481e89c (127.7.213.67) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 3 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:14.180320 16299 raft_consensus.cc:1275] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.181447 17198 consensus_queue.cc:1048] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.191666 16225 raft_consensus.cc:1275] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.194479 17187 consensus_queue.cc:1048] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:14.204921 16299 raft_consensus.cc:1275] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.206938 17187 consensus_queue.cc:1048] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.226406 16225 raft_consensus.cc:1275] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.228048 17186 consensus_queue.cc:1048] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.274492 16225 raft_consensus.cc:1275] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer 115c2d35eda349178144f8365481e89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.275651 17177 consensus_queue.cc:1048] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.285651 16372 raft_consensus.cc:1275] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer 115c2d35eda349178144f8365481e89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.287572 17211 consensus_queue.cc:1048] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:14.308306 16225 raft_consensus.cc:1275] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.309615 17187 consensus_queue.cc:1048] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.322732 16372 raft_consensus.cc:1275] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.324024 17186 consensus_queue.cc:1048] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.536620 16299 raft_consensus.cc:1275] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:14.537855 17186 consensus_queue.cc:1048] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:14.547428 16372 raft_consensus.cc:1275] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 3 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:14.548892 17186 consensus_queue.cc:1048] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:14.565879 16219 raft_consensus.cc:1275] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 3 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:14.567066 17186 consensus_queue.cc:1048] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:14.582705 16299 raft_consensus.cc:1275] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:14.584233 17187 consensus_queue.cc:1048] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:14.600986  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:14.650473  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:14.701167 16372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626"
dest_uuid: "c8106927ed954e1c800869427b6c7964"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:40076
I20260709 13:07:14.701928 16372 raft_consensus.cc:606] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:14.704401 16225 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c"
dest_uuid: "3ce6da6d71124256be0679451014f776"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:55110
I20260709 13:07:14.704932 16225 raft_consensus.cc:606] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:14.707468 16299 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee"
dest_uuid: "115c2d35eda349178144f8365481e89c"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:44392
I20260709 13:07:14.707958 16299 raft_consensus.cc:606] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:14.710451 16225 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359"
dest_uuid: "3ce6da6d71124256be0679451014f776"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:55110
I20260709 13:07:14.711045 16225 raft_consensus.cc:606] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:14.713967 16372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d"
dest_uuid: "c8106927ed954e1c800869427b6c7964"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:40076
I20260709 13:07:14.714504 16372 raft_consensus.cc:606] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:14.715631  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:14.717232  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:14.828183 17211 raft_consensus.cc:993] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:14.828830 17211 raft_consensus.cc:1081] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:14.830785 17033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45066
I20260709 13:07:14.831357 17033 raft_consensus.cc:493] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.831732 17033 raft_consensus.cc:3060] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.838321 17033 raft_consensus.cc:515] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:14.840021 17180 raft_consensus.cc:993] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:14.840576 17168 raft_consensus.cc:1081] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:14.840880 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:14.841497 16298 raft_consensus.cc:3055] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.841805 16298 raft_consensus.cc:740] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:14.842581 16298 consensus_queue.cc:260] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:14.843603 16298 raft_consensus.cc:3060] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.844440 17024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45078
I20260709 13:07:14.845001 17024 raft_consensus.cc:493] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.845389 17024 raft_consensus.cc:3060] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.848469 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f529392f494e6d83be5434d71ed0ee" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:14.849270 16372 raft_consensus.cc:3060] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.851718 17033 leader_election.cc:290] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:14.854336 17024 raft_consensus.cc:515] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:14.858209 17024 leader_election.cc:290] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:14.859849 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:14.860421 16370 raft_consensus.cc:3055] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.860740 16370 raft_consensus.cc:740] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:14.861524 16298 raft_consensus.cc:2468] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:14.861518 16370 consensus_queue.cc:260] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:14.862881 16951 leader_election.cc:304] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:07:14.863379 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7d234ecf1f47fabae92b3be1d2b84d" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:14.863669 17186 raft_consensus.cc:2804] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:14.863168 16370 raft_consensus.cc:3060] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.864167 16299 raft_consensus.cc:3060] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.864310 17186 raft_consensus.cc:697] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:14.865442 17186 consensus_queue.cc:237] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:14.869397 16372 raft_consensus.cc:2468] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:14.873211 16370 raft_consensus.cc:2468] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:14.874050 16299 raft_consensus.cc:2468] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:14.874146 16950 leader_election.cc:304] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:14.875147 17198 raft_consensus.cc:2804] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:14.875674 17198 raft_consensus.cc:697] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:14.876683 17198 consensus_queue.cc:237] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:14.882517 16048 catalog_manager.cc:5697] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from 115c2d35eda349178144f8365481e89c (127.7.213.67) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:14.907152 16048 catalog_manager.cc:5697] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from c8106927ed954e1c800869427b6c7964 (127.7.213.68) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:14.966889 17172 raft_consensus.cc:993] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:14.967459 17176 raft_consensus.cc:1081] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:14.969238 17024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45082
I20260709 13:07:14.969794 17024 raft_consensus.cc:493] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:14.970074 17024 raft_consensus.cc:3060] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.975170 17024 raft_consensus.cc:515] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:14.976975 17024 leader_election.cc:290] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:14.977605 16370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:14.977972 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c751e8904a94d8286347a543fbbc90c" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:14.978271 16370 raft_consensus.cc:3060] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.978562 16219 raft_consensus.cc:3055] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:14.978880 16219 raft_consensus.cc:740] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:07:14.979657 16219 consensus_queue.cc:260] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:14.980911 16219 raft_consensus.cc:3060] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:14.985883 16370 raft_consensus.cc:2468] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:14.987017 16950 leader_election.cc:304] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:14.987866 17198 raft_consensus.cc:2804] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:14.988296 17198 raft_consensus.cc:697] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:14.989074 17198 consensus_queue.cc:237] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:14.991397 16219 raft_consensus.cc:2468] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:14.998356 16048 catalog_manager.cc:5697] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from 3ce6da6d71124256be0679451014f776 (127.7.213.66) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:15.098776 17180 raft_consensus.cc:993] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:15.099237 17168 raft_consensus.cc:1081] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:15.100600 17033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45078
I20260709 13:07:15.101050 17033 raft_consensus.cc:493] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.101328 17033 raft_consensus.cc:3060] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.105792 17033 raft_consensus.cc:515] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:15.108318 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:15.108592 17033 leader_election.cc:290] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:15.108950 16372 raft_consensus.cc:3055] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.109891 16372 raft_consensus.cc:740] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:15.111304 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0255ba6a147b416d8e3081a4e28bf626" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:15.111284 16372 consensus_queue.cc:260] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:15.111953 16219 raft_consensus.cc:3060] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.112747 16372 raft_consensus.cc:3060] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.117887 16219 raft_consensus.cc:2468] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:15.118458 16372 raft_consensus.cc:2468] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:15.118954 16950 leader_election.cc:304] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:07:15.119825 17186 raft_consensus.cc:2804] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.120330 17186 raft_consensus.cc:697] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:15.121173 17186 consensus_queue.cc:237] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:15.130780 16048 catalog_manager.cc:5697] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from c8106927ed954e1c800869427b6c7964 (127.7.213.68) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:15.172345 17172 raft_consensus.cc:993] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:15.172851 17172 raft_consensus.cc:1081] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:15.174373 17033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45082
I20260709 13:07:15.174916 17033 raft_consensus.cc:493] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.175240 17033 raft_consensus.cc:3060] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.181121 17033 raft_consensus.cc:515] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:15.185707 17033 leader_election.cc:290] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:15.184963 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:15.186187 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b90ea5d39bbf4d79a4a44959a8f40359" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:15.186529 16372 raft_consensus.cc:3060] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.186792 16225 raft_consensus.cc:3055] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.187120 16225 raft_consensus.cc:740] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ce6da6d71124256be0679451014f776, State: Running, Role: LEADER
I20260709 13:07:15.187944 16225 consensus_queue.cc:260] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:15.189110 16225 raft_consensus.cc:3060] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.194331 16372 raft_consensus.cc:2468] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:15.195493 16950 leader_election.cc:304] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:15.196574 17186 raft_consensus.cc:2804] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.197088 17186 raft_consensus.cc:697] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:15.198169 16225 raft_consensus.cc:2468] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:15.198010 17186 consensus_queue.cc:237] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:15.208102 16048 catalog_manager.cc:5697] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from 3ce6da6d71124256be0679451014f776 (127.7.213.66) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:15.260301 16299 raft_consensus.cc:1275] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.261480 17186 consensus_queue.cc:1048] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.271584 16370 raft_consensus.cc:1275] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.273368 17186 consensus_queue.cc:1048] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.282207 16299 raft_consensus.cc:1275] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.283932 17186 consensus_queue.cc:1048] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.295079 16370 raft_consensus.cc:1275] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.296926 17186 consensus_queue.cc:1048] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.483790 16225 raft_consensus.cc:1275] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.485037 17186 consensus_queue.cc:1048] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.496445 16370 raft_consensus.cc:1275] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.498780 17186 consensus_queue.cc:1048] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:15.576570 16370 raft_consensus.cc:1275] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.577800 17187 consensus_queue.cc:1048] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.586833 16225 raft_consensus.cc:1275] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.589458 17198 consensus_queue.cc:1048] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.718101  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:15.740808 16372 raft_consensus.cc:1275] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.742724 17186 consensus_queue.cc:1048] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.751003 16225 raft_consensus.cc:1275] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:15.752888 17187 consensus_queue.cc:1048] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:15.772014  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:15.826717 16372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "819e82eed43e4968984673887beb39ec"
dest_uuid: "c8106927ed954e1c800869427b6c7964"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:40076
I20260709 13:07:15.827309 16372 raft_consensus.cc:606] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:15.829843 16372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a"
dest_uuid: "c8106927ed954e1c800869427b6c7964"
mode: GRACEFUL
new_leader_uuid: "115c2d35eda349178144f8365481e89c"
 from {username='slave'} at 127.0.0.1:40076
I20260709 13:07:15.830355 16372 raft_consensus.cc:606] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Received request to transfer leadership to TS 115c2d35eda349178144f8365481e89c
I20260709 13:07:15.832870 16372 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2"
dest_uuid: "c8106927ed954e1c800869427b6c7964"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:40076
I20260709 13:07:15.833446 16372 raft_consensus.cc:606] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:15.834601  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:15.835623  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:15.903172 17180 raft_consensus.cc:993] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:15.903685 17168 raft_consensus.cc:1081] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:15.904955 17024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45078
I20260709 13:07:15.905341 17024 raft_consensus.cc:493] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.905573 17024 raft_consensus.cc:3060] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.910557 17024 raft_consensus.cc:515] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:15.912360 17024 leader_election.cc:290] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791), 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495)
I20260709 13:07:15.913252 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:15.913635 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af66e3a8024e47fe925b7afad68d41c2" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:15.913831 16372 raft_consensus.cc:3055] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:15.914188 16225 raft_consensus.cc:3060] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.914201 16372 raft_consensus.cc:740] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:15.915066 16372 consensus_queue.cc:260] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:15.916082 16372 raft_consensus.cc:3060] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:15.919061 16225 raft_consensus.cc:2468] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:15.920010 16950 leader_election.cc:304] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce6da6d71124256be0679451014f776, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:07:15.920655 16372 raft_consensus.cc:2468] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:15.920706 17187 raft_consensus.cc:2804] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:15.921382 17187 raft_consensus.cc:697] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:15.922230 17187 consensus_queue.cc:237] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } }
I20260709 13:07:15.931423 16048 catalog_manager.cc:5697] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from c8106927ed954e1c800869427b6c7964 (127.7.213.68) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:15.977298 17180 raft_consensus.cc:993] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:15.977845 17168 raft_consensus.cc:1081] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:15.979367 17024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "819e82eed43e4968984673887beb39ec"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45078
I20260709 13:07:15.980019 17024 raft_consensus.cc:493] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:15.980369 17024 raft_consensus.cc:3060] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.985308 17024 raft_consensus.cc:515] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:15.986750 17024 leader_election.cc:290] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 3 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:15.987399 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:15.987931 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "819e82eed43e4968984673887beb39ec" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:15.988085 16299 raft_consensus.cc:3060] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.988535 16372 raft_consensus.cc:3055] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:15.988868 16372 raft_consensus.cc:740] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:15.989567 16372 consensus_queue.cc:260] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:15.990492 16372 raft_consensus.cc:3060] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:15.992709 16299 raft_consensus.cc:2468] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 3.
I20260709 13:07:15.993858 16951 leader_election.cc:304] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d; no voters: 
I20260709 13:07:15.994585 17187 raft_consensus.cc:2804] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:15.995028 17187 raft_consensus.cc:697] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 3 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:15.996690 17187 consensus_queue.cc:237] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:15.998802 16372 raft_consensus.cc:2468] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 3.
I20260709 13:07:16.009855 16048 catalog_manager.cc:5697] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 2 to 3, leader changed from c8106927ed954e1c800869427b6c7964 (127.7.213.68) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 3 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:16.117017 17249 raft_consensus.cc:993] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964: : Instructing follower 115c2d35eda349178144f8365481e89c to start an election
I20260709 13:07:16.118685 17180 raft_consensus.cc:1081] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Signalling peer 115c2d35eda349178144f8365481e89c to start an election
I20260709 13:07:16.120908 16299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a"
dest_uuid: "115c2d35eda349178144f8365481e89c"
 from {username='slave'} at 127.0.0.1:37816
I20260709 13:07:16.121487 16299 raft_consensus.cc:493] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:16.121851 16299 raft_consensus.cc:3060] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.127182 16299 raft_consensus.cc:515] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:07:16.129024 16299 leader_election.cc:290] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Requested vote from peers 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:16.129585 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ce6da6d71124256be0679451014f776"
I20260709 13:07:16.129995 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a58bd7b2451404a8b70547dee94049a" candidate_uuid: "115c2d35eda349178144f8365481e89c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:16.130220 16225 raft_consensus.cc:3060] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.131965 16372 raft_consensus.cc:3055] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:16.132295 16372 raft_consensus.cc:740] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c8106927ed954e1c800869427b6c7964, State: Running, Role: LEADER
I20260709 13:07:16.132982 16372 consensus_queue.cc:260] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:07:16.134277 16372 raft_consensus.cc:3060] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.135533 16225 raft_consensus.cc:2468] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 2.
I20260709 13:07:16.137118 16262 leader_election.cc:304] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 115c2d35eda349178144f8365481e89c, 3ce6da6d71124256be0679451014f776; no voters: 
I20260709 13:07:16.138932 17185 raft_consensus.cc:2804] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:16.139457 17185 raft_consensus.cc:697] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Becoming Leader. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:16.140532 17185 consensus_queue.cc:237] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } }
I20260709 13:07:16.141652 16372 raft_consensus.cc:2468] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 115c2d35eda349178144f8365481e89c in term 2.
I20260709 13:07:16.152129 16048 catalog_manager.cc:5697] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c reported cstate change: term changed from 1 to 2, leader changed from c8106927ed954e1c800869427b6c7964 (127.7.213.68) to 115c2d35eda349178144f8365481e89c (127.7.213.67). New cstate: current_term: 2 leader_uuid: "115c2d35eda349178144f8365481e89c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:16.349612 16372 raft_consensus.cc:1275] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.350798 17187 consensus_queue.cc:1048] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.360954 16225 raft_consensus.cc:1275] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.362600 17186 consensus_queue.cc:1048] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:16.384212 16370 raft_consensus.cc:1275] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 3 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:16.385716 17198 consensus_queue.cc:1048] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:16.400056 16299 raft_consensus.cc:1275] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:16.402961 17259 consensus_queue.cc:1048] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:16.724583 16220 raft_consensus.cc:1275] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Refusing update from remote peer 115c2d35eda349178144f8365481e89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.725734 17185 consensus_queue.cc:1048] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce6da6d71124256be0679451014f776" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.735519 16370 raft_consensus.cc:1275] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer 115c2d35eda349178144f8365481e89c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:16.736954 17211 consensus_queue.cc:1048] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.836592  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:16.883662  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:16.920068 16298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0"
dest_uuid: "115c2d35eda349178144f8365481e89c"
mode: GRACEFUL
new_leader_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:44392
I20260709 13:07:16.920691 16298 raft_consensus.cc:606] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Received request to transfer leadership to TS aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:16.921774  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:07:16.922823  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:16.950587 17185 raft_consensus.cc:993] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c: : Instructing follower aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:16.951467 17250 raft_consensus.cc:1081] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Signalling peer aa8eb07867e7420b80a7e615d570509d to start an election
I20260709 13:07:16.952912 17024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0"
dest_uuid: "aa8eb07867e7420b80a7e615d570509d"
 from {username='slave'} at 127.0.0.1:45066
I20260709 13:07:16.953449 17024 raft_consensus.cc:493] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:16.953785 17024 raft_consensus.cc:3060] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.958233 17024 raft_consensus.cc:515] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:16.959688 17024 leader_election.cc:290] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Requested vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), c8106927ed954e1c800869427b6c7964 (127.7.213.68:36791)
I20260709 13:07:16.960587 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "115c2d35eda349178144f8365481e89c"
I20260709 13:07:16.960587 16372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72031807a36e49fa81fe0eb9fb94a7a0" candidate_uuid: "aa8eb07867e7420b80a7e615d570509d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c8106927ed954e1c800869427b6c7964"
I20260709 13:07:16.961401 16298 raft_consensus.cc:3055] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:16.961450 16372 raft_consensus.cc:3060] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.961804 16298 raft_consensus.cc:740] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 115c2d35eda349178144f8365481e89c, State: Running, Role: LEADER
I20260709 13:07:16.962579 16298 consensus_queue.cc:260] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:16.963809 16298 raft_consensus.cc:3060] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:16.968139 16372 raft_consensus.cc:2468] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:16.969769 16950 leader_election.cc:304] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa8eb07867e7420b80a7e615d570509d, c8106927ed954e1c800869427b6c7964; no voters: 
I20260709 13:07:16.970597 17198 raft_consensus.cc:2804] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:16.971065 17198 raft_consensus.cc:697] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Becoming Leader. State: Replica: aa8eb07867e7420b80a7e615d570509d, State: Running, Role: LEADER
I20260709 13:07:16.971774 16298 raft_consensus.cc:2468] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa8eb07867e7420b80a7e615d570509d in term 2.
I20260709 13:07:16.971958 17198 consensus_queue.cc:237] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:16.981560 16045 catalog_manager.cc:5697] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d reported cstate change: term changed from 1 to 2, leader changed from 115c2d35eda349178144f8365481e89c (127.7.213.67) to aa8eb07867e7420b80a7e615d570509d (127.7.213.65). New cstate: current_term: 2 leader_uuid: "aa8eb07867e7420b80a7e615d570509d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.572351 16372 raft_consensus.cc:1275] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:17.573719 17259 consensus_queue.cc:1048] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:17.583333 16299 raft_consensus.cc:1275] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Refusing update from remote peer aa8eb07867e7420b80a7e615d570509d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:17.590598 17198 consensus_queue.cc:1048] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [LEADER]: Connected to new peer: Peer: permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:17.923712  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:17.960955  8021 tablet_server.cc:179] TabletServer@127.7.213.65:40593 shutting down...
W20260709 13:07:17.986334 16190 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 257 similar messages]
W20260709 13:07:17.991566 16190 consensus_peers.cc:597] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:17.999051  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:18.000258  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.001714  8021 raft_consensus.cc:2243] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.002224  8021 raft_consensus.cc:2272] T 47e0028829c94c99a91ca4cc4d629a4a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.005458  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.006188  8021 raft_consensus.cc:2243] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.006721  8021 raft_consensus.cc:2272] T 76774e4153c7483683fcec5f557bfaad P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.009608  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.010273  8021 raft_consensus.cc:2243] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:18.010309 16338 consensus_peers.cc:597] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.010797  8021 raft_consensus.cc:2272] T c14505e666fc47d29cfb0c9bc323384e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.013060  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.013679  8021 raft_consensus.cc:2243] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.014225  8021 raft_consensus.cc:2272] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.016745  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.017335  8021 raft_consensus.cc:2243] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.019362  8021 raft_consensus.cc:2272] T bd508336a7c844cc84117b7ef3a565ba P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.021890  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.022523  8021 raft_consensus.cc:2243] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.023350  8021 raft_consensus.cc:2272] T 58b7aa0c7f8b4bf4a2e4881b23351045 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.026000  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.026870  8021 raft_consensus.cc:2243] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:07:18.027168 16190 consensus_peers.cc:597] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.028044  8021 raft_consensus.cc:2272] T 2c751e8904a94d8286347a543fbbc90c P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.030243  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.030752  8021 raft_consensus.cc:2243] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.031517  8021 raft_consensus.cc:2272] T b90ea5d39bbf4d79a4a44959a8f40359 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.033964  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.034966  8021 raft_consensus.cc:2243] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.035455  8021 raft_consensus.cc:2272] T 3c337868c6fc4e079f20a204e451ab66 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.037690 16190 consensus_peers.cc:597] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.038214  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.038888  8021 raft_consensus.cc:2243] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.039292  8021 raft_consensus.cc:2272] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.041675  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.042384  8021 raft_consensus.cc:2243] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.042892  8021 raft_consensus.cc:2272] T c66825803ddc46e8849c7718d3cb56ea P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.045595  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.046432  8021 raft_consensus.cc:2243] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.046902  8021 raft_consensus.cc:2272] T a6563b9b2e954646b0ce2535138676d0 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.048328 16338 consensus_peers.cc:597] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.049187  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.049858  8021 raft_consensus.cc:2243] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.050575  8021 raft_consensus.cc:2272] T 8f5d69137b6e468bb06eb4212b198f84 P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.052721  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.053344  8021 raft_consensus.cc:2243] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.054039  8021 raft_consensus.cc:2272] T 18645784203b4ce1bbe3302513d910c1 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.056002  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.056717  8021 raft_consensus.cc:2243] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.057479  8021 raft_consensus.cc:2272] T de7d234ecf1f47fabae92b3be1d2b84d P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.059314  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.059895  8021 raft_consensus.cc:2243] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.060365  8021 raft_consensus.cc:2272] T 42c540d71b564c08b9032559d7975cc5 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.062454  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.063169  8021 raft_consensus.cc:2243] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.063571  8021 raft_consensus.cc:2272] T 62345643f7684ec08f60f740de27cead P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.065786  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.066543  8021 raft_consensus.cc:2243] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.066929  8021 raft_consensus.cc:2272] T 1dd94eae6c1241e5838930e00eb43c4c P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.069540  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.070302  8021 raft_consensus.cc:2243] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.070714  8021 raft_consensus.cc:2272] T 361bd03c04594782911afd84f865c9f5 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.072834  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.073390  8021 raft_consensus.cc:2243] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.073853  8021 raft_consensus.cc:2272] T 7973b570c8554e048bd5a9d02fd13a46 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.076674  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.077452  8021 raft_consensus.cc:2243] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.078235  8021 raft_consensus.cc:2272] T 44f529392f494e6d83be5434d71ed0ee P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.080129  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.080928  8021 raft_consensus.cc:2243] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.081316  8021 raft_consensus.cc:2272] T 4ba3cb60361b431ba89f7842cdb1c85a P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.083981  8021 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:18.084251 16338 consensus_peers.cc:597] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.084733  8021 raft_consensus.cc:2243] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.085333  8021 raft_consensus.cc:2272] T 38bbd6b3bdff467f9d48329073cf0167 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.088042  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.088593  8021 raft_consensus.cc:2243] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.089112  8021 raft_consensus.cc:2272] T 55b5e6c6090448ab831f8afeef43db0d P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.091801  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.092590  8021 raft_consensus.cc:2243] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.093103  8021 raft_consensus.cc:2272] T 2201023e16254eed9712e5fba24caa38 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.095510  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.096175  8021 raft_consensus.cc:2243] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.096621  8021 raft_consensus.cc:2272] T 181b2ad7f8b649d4be93241715d03572 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.097395 16190 consensus_peers.cc:597] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.098783  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.099435  8021 raft_consensus.cc:2243] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.100207  8021 raft_consensus.cc:2272] T 829bf55d32c344af8ae546eb15903faa P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.102175  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.102730  8021 raft_consensus.cc:2243] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.103137  8021 raft_consensus.cc:2272] T f603bf0f99d0428194f0998267e8713d P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.105499  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.106052  8021 raft_consensus.cc:2243] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.106916  8021 raft_consensus.cc:2272] T 0255ba6a147b416d8e3081a4e28bf626 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.108871  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.109349  8021 raft_consensus.cc:2243] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.110064  8021 raft_consensus.cc:2272] T 09715ac5d7d741e3aabf462dcab633ba P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.111872  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.112334  8021 raft_consensus.cc:2243] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.112658  8021 raft_consensus.cc:2272] T a6959f2cf9714e91aee31bf53776915b P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.114416  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.115013  8021 raft_consensus.cc:2243] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.115902  8021 raft_consensus.cc:2272] T af66e3a8024e47fe925b7afad68d41c2 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.118172  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.118821  8021 raft_consensus.cc:2243] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.119252  8021 raft_consensus.cc:2272] T e9db6747820949b6b76dcc3ab05485b1 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.121822  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.122355  8021 raft_consensus.cc:2243] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.123174  8021 raft_consensus.cc:2272] T 72031807a36e49fa81fe0eb9fb94a7a0 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.126139  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.126940  8021 raft_consensus.cc:2243] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.127528  8021 raft_consensus.cc:2272] T f228d4a700de40ada51433b8043bf6a7 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.130026  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.130607  8021 raft_consensus.cc:2243] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.131016  8021 raft_consensus.cc:2272] T 2128e5983b684efeacc628400afa861f P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.133375  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.133960  8021 raft_consensus.cc:2243] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.134486  8021 raft_consensus.cc:2272] T 26c52cafca664c5f90589901f62e4b16 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.137004  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.137676  8021 raft_consensus.cc:2243] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.138188  8021 raft_consensus.cc:2272] T c0f16aeeb02645aea6c44a180eac4f87 P aa8eb07867e7420b80a7e615d570509d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.140393  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.140942  8021 raft_consensus.cc:2243] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.141731  8021 raft_consensus.cc:2272] T 819e82eed43e4968984673887beb39ec P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.144331  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.144972  8021 raft_consensus.cc:2243] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.145402  8021 raft_consensus.cc:2272] T 3699132ec3bb4f8baa0204a17bda51f9 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.147632  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.148279  8021 raft_consensus.cc:2243] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.148763  8021 raft_consensus.cc:2272] T a042ac05958049c38414fa65d0c39414 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.151034  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.151578  8021 raft_consensus.cc:2243] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.152052  8021 raft_consensus.cc:2272] T 6714e2c053f14fbeaec12683da401eb4 P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.154721  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.155366  8021 raft_consensus.cc:2243] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.155916  8021 raft_consensus.cc:2272] T abb840ca52d04206a82103fb4eb79ccf P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.158581  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.159216  8021 raft_consensus.cc:2243] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.160192  8021 raft_consensus.cc:2272] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P aa8eb07867e7420b80a7e615d570509d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.162794  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:18.163314  8021 raft_consensus.cc:2243] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.163707  8021 raft_consensus.cc:2272] T 1047f65a57d24cafbf3f41f5b235556e P aa8eb07867e7420b80a7e615d570509d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.178594 16338 consensus_peers.cc:597] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.186139 16264 consensus_peers.cc:597] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.207656 16190 consensus_peers.cc:597] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.207981 16264 consensus_peers.cc:597] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.230873 16190 consensus_peers.cc:597] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.237363  8021 tablet_server.cc:196] TabletServer@127.7.213.65:40593 shutdown complete.
W20260709 13:07:18.246378 16190 consensus_peers.cc:597] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.248946 16264 consensus_peers.cc:597] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.256338 16264 consensus_peers.cc:597] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.262020 16338 consensus_peers.cc:597] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.271620 16190 consensus_peers.cc:597] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.297066 16190 consensus_peers.cc:597] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.306032 16338 consensus_peers.cc:597] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.318919 16264 consensus_peers.cc:597] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.325088  8021 tablet_server.cc:179] TabletServer@127.7.213.66:0 shutting down...
W20260709 13:07:18.336287 16264 consensus_peers.cc:597] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.354120 16264 consensus_peers.cc:597] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.359691 16190 consensus_peers.cc:597] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.359817 16338 consensus_peers.cc:597] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.363502 16336 consensus_peers.cc:597] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.369385 16190 consensus_peers.cc:597] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.377363  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:18.378401  8021 tablet_replica.cc:333] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.379443  8021 raft_consensus.cc:2243] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.380146  8021 raft_consensus.cc:2272] T 6714e2c053f14fbeaec12683da401eb4 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.381668 16336 consensus_peers.cc:597] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.382781  8021 tablet_replica.cc:333] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.383503  8021 raft_consensus.cc:2243] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.384354  8021 raft_consensus.cc:2272] T a6959f2cf9714e91aee31bf53776915b P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.386397  8021 tablet_replica.cc:333] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.386950  8021 raft_consensus.cc:2243] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.387374  8021 raft_consensus.cc:2272] T af66e3a8024e47fe925b7afad68d41c2 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.389596  8021 tablet_replica.cc:333] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.390149  8021 raft_consensus.cc:2243] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.390547  8021 raft_consensus.cc:2272] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.392776  8021 tablet_replica.cc:333] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.393560  8021 raft_consensus.cc:2243] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.394107  8021 raft_consensus.cc:2272] T 424dbaa8a798455b9ae4d2f0b0989561 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.396806  8021 tablet_replica.cc:333] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.397632  8021 raft_consensus.cc:2243] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.398218  8021 raft_consensus.cc:2272] T 1047f65a57d24cafbf3f41f5b235556e P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.400857  8021 tablet_replica.cc:333] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.401638  8021 raft_consensus.cc:2243] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.402191  8021 raft_consensus.cc:2272] T 4ba3cb60361b431ba89f7842cdb1c85a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.404745  8021 tablet_replica.cc:333] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.405551  8021 raft_consensus.cc:2243] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.406119  8021 raft_consensus.cc:2272] T 1dea723251ed4848b42709f873b7d446 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.408085  8021 tablet_replica.cc:333] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.408731  8021 raft_consensus.cc:2243] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.409108  8021 raft_consensus.cc:2272] T 4cec4e308d104a9f9049fa0c2263b58d P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.411221  8021 tablet_replica.cc:333] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.411878  8021 raft_consensus.cc:2243] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.412251  8021 raft_consensus.cc:2272] T 09715ac5d7d741e3aabf462dcab633ba P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.414086  8021 tablet_replica.cc:333] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.414681  8021 raft_consensus.cc:2243] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.415172  8021 raft_consensus.cc:2272] T 0255ba6a147b416d8e3081a4e28bf626 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.416993  8021 tablet_replica.cc:333] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.417582  8021 raft_consensus.cc:2243] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.418378  8021 raft_consensus.cc:2272] T 7973b570c8554e048bd5a9d02fd13a46 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.420370  8021 tablet_replica.cc:333] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.421063  8021 raft_consensus.cc:2243] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.421490  8021 raft_consensus.cc:2272] T abb840ca52d04206a82103fb4eb79ccf P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.423664  8021 tablet_replica.cc:333] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.424248  8021 raft_consensus.cc:2243] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.424690  8021 raft_consensus.cc:2272] T 42c540d71b564c08b9032559d7975cc5 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.426726  8021 tablet_replica.cc:333] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.427233  8021 raft_consensus.cc:2243] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.427779  8021 raft_consensus.cc:2272] T 1dd94eae6c1241e5838930e00eb43c4c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.430256  8021 tablet_replica.cc:333] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.430840  8021 raft_consensus.cc:2243] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.431887  8021 raft_consensus.cc:2272] T 62345643f7684ec08f60f740de27cead P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.432137 16262 consensus_peers.cc:597] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.434484  8021 tablet_replica.cc:333] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.435436  8021 raft_consensus.cc:2243] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.436069  8021 raft_consensus.cc:2272] T 7186244ab72e4b32ad068941b913f052 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.438899  8021 tablet_replica.cc:333] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.439735  8021 raft_consensus.cc:2243] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.440299  8021 raft_consensus.cc:2272] T f228d4a700de40ada51433b8043bf6a7 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.442484  8021 tablet_replica.cc:333] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.443001  8021 raft_consensus.cc:2243] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.443735  8021 raft_consensus.cc:2272] T e9db6747820949b6b76dcc3ab05485b1 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.445721  8021 tablet_replica.cc:333] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.446252  8021 raft_consensus.cc:2243] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.446640  8021 raft_consensus.cc:2272] T b0d1691a17794035a9fae371cbabb4be P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.448514  8021 tablet_replica.cc:333] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.449072  8021 raft_consensus.cc:2243] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.449942  8021 raft_consensus.cc:2272] T 3c337868c6fc4e079f20a204e451ab66 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.451920  8021 tablet_replica.cc:333] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.452513  8021 raft_consensus.cc:2243] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.452939  8021 raft_consensus.cc:2272] T 31991d68fa8642189423a50030c4e7d5 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.454749  8021 tablet_replica.cc:333] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.455253  8021 raft_consensus.cc:2243] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.456097  8021 raft_consensus.cc:2272] T a042ac05958049c38414fa65d0c39414 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.458024  8021 tablet_replica.cc:333] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.458709  8021 raft_consensus.cc:2243] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.459118  8021 raft_consensus.cc:2272] T e652cb454b674e399c2ac22c22c05506 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.460994  8021 tablet_replica.cc:333] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.461578  8021 raft_consensus.cc:2243] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.461992  8021 raft_consensus.cc:2272] T c14505e666fc47d29cfb0c9bc323384e P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.463722  8021 tablet_replica.cc:333] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.464315  8021 raft_consensus.cc:2243] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.464735  8021 raft_consensus.cc:2272] T 8f5d69137b6e468bb06eb4212b198f84 P 3ce6da6d71124256be0679451014f776 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.466674  8021 tablet_replica.cc:333] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.467217  8021 raft_consensus.cc:2243] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.468037  8021 raft_consensus.cc:2272] T b521a909fd464b72bc2742ff3157658b P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.470151  8021 tablet_replica.cc:333] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.470785  8021 raft_consensus.cc:2243] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.471203  8021 raft_consensus.cc:2272] T c0cbd7d8de354586aab3116154d3ee3c P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.473740  8021 tablet_replica.cc:333] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.474530  8021 raft_consensus.cc:2243] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.474993  8021 raft_consensus.cc:2272] T 2c751e8904a94d8286347a543fbbc90c P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.476864  8021 tablet_replica.cc:333] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.477363  8021 raft_consensus.cc:2243] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.477722  8021 raft_consensus.cc:2272] T b90ea5d39bbf4d79a4a44959a8f40359 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.479411  8021 tablet_replica.cc:333] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.480020  8021 raft_consensus.cc:2243] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.480846  8021 raft_consensus.cc:2272] T 26c52cafca664c5f90589901f62e4b16 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.482965  8021 tablet_replica.cc:333] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.483436  8021 raft_consensus.cc:2243] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.483834  8021 raft_consensus.cc:2272] T 2128e5983b684efeacc628400afa861f P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.485549  8021 tablet_replica.cc:333] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.485989  8021 raft_consensus.cc:2243] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.486683  8021 raft_consensus.cc:2272] T 76774e4153c7483683fcec5f557bfaad P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.488564  8021 tablet_replica.cc:333] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.489063  8021 raft_consensus.cc:2243] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.490028  8021 raft_consensus.cc:2272] T ceba3482809245aa87c4e743269df937 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.492110  8021 tablet_replica.cc:333] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776: stopping tablet replica
W20260709 13:07:18.492789 16264 consensus_peers.cc:597] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.492796  8021 raft_consensus.cc:2243] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.493882  8021 raft_consensus.cc:2272] T c66825803ddc46e8849c7718d3cb56ea P 3ce6da6d71124256be0679451014f776 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.495707  8021 tablet_replica.cc:333] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.496272  8021 raft_consensus.cc:2243] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.496717  8021 raft_consensus.cc:2272] T bd508336a7c844cc84117b7ef3a565ba P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.498941  8021 tablet_replica.cc:333] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.499503  8021 raft_consensus.cc:2243] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.499953  8021 raft_consensus.cc:2272] T 829bf55d32c344af8ae546eb15903faa P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.501858  8021 tablet_replica.cc:333] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.502405  8021 raft_consensus.cc:2243] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.503288  8021 raft_consensus.cc:2272] T f603bf0f99d0428194f0998267e8713d P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.506042  8021 tablet_replica.cc:333] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
W20260709 13:07:18.506134 16336 consensus_peers.cc:597] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.506757  8021 raft_consensus.cc:2243] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.507771  8021 raft_consensus.cc:2272] T 2201023e16254eed9712e5fba24caa38 P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.508656 16338 consensus_peers.cc:597] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.509902  8021 tablet_replica.cc:333] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.510462  8021 raft_consensus.cc:2243] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.510860  8021 raft_consensus.cc:2272] T ffb9eaed164d46ccac187f1878f24e4a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.512789  8021 tablet_replica.cc:333] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.513285  8021 raft_consensus.cc:2243] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.514011  8021 raft_consensus.cc:2272] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.516199  8021 tablet_replica.cc:333] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.516712  8021 raft_consensus.cc:2243] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.517108  8021 raft_consensus.cc:2272] T 7e218598a3114f679b2483143a2f4692 P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.518997  8021 tablet_replica.cc:333] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.519455  8021 raft_consensus.cc:2243] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.519896  8021 raft_consensus.cc:2272] T 8a58bd7b2451404a8b70547dee94049a P 3ce6da6d71124256be0679451014f776 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.521618  8021 tablet_replica.cc:333] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776: stopping tablet replica
I20260709 13:07:18.522042  8021 raft_consensus.cc:2243] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.522874  8021 raft_consensus.cc:2272] T 47e0028829c94c99a91ca4cc4d629a4a P 3ce6da6d71124256be0679451014f776 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.535259 16338 consensus_peers.cc:597] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.536201 16262 consensus_peers.cc:597] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.542963 16336 consensus_peers.cc:597] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.545429 16336 consensus_peers.cc:597] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.565207 16264 consensus_peers.cc:597] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.566475 16336 consensus_peers.cc:597] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.567611 16264 consensus_peers.cc:597] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c -> Peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Couldn't send request to peer aa8eb07867e7420b80a7e615d570509d. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.570856 16336 consensus_peers.cc:597] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.582687 16262 consensus_peers.cc:597] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.605340 16336 consensus_peers.cc:597] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.615056  8021 tablet_server.cc:196] TabletServer@127.7.213.66:0 shutdown complete.
W20260709 13:07:18.630146 16262 consensus_peers.cc:597] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.636153 16262 consensus_peers.cc:597] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.699218  8021 tablet_server.cc:179] TabletServer@127.7.213.67:0 shutting down...
W20260709 13:07:18.709746 16337 consensus_peers.cc:597] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.714761 16337 consensus_peers.cc:597] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.724536 16337 consensus_peers.cc:597] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.746390 16336 consensus_peers.cc:597] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.749080  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:18.750007  8021 tablet_replica.cc:333] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.750681  8021 raft_consensus.cc:2243] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:18.750624 16262 consensus_peers.cc:597] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.751395 16262 consensus_peers.cc:597] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.751520  8021 raft_consensus.cc:2272] T c0cbd7d8de354586aab3116154d3ee3c P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.751981 16262 consensus_peers.cc:597] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.754273  8021 tablet_replica.cc:333] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.754992  8021 raft_consensus.cc:2243] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.756037  8021 raft_consensus.cc:2272] T 6714e2c053f14fbeaec12683da401eb4 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.758097  8021 tablet_replica.cc:333] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.758639  8021 raft_consensus.cc:2243] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.759063  8021 raft_consensus.cc:2272] T 7186244ab72e4b32ad068941b913f052 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.760972  8021 tablet_replica.cc:333] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.761505  8021 raft_consensus.cc:2243] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.762287  8021 raft_consensus.cc:2272] T 38bbd6b3bdff467f9d48329073cf0167 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.764200  8021 tablet_replica.cc:333] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.764747  8021 raft_consensus.cc:2243] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.765149  8021 raft_consensus.cc:2272] T 58b7aa0c7f8b4bf4a2e4881b23351045 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.767153  8021 tablet_replica.cc:333] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.767896  8021 raft_consensus.cc:2243] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.768275  8021 raft_consensus.cc:2272] T 76774e4153c7483683fcec5f557bfaad P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.770960  8021 tablet_replica.cc:333] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.771780  8021 raft_consensus.cc:2243] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.772239  8021 raft_consensus.cc:2272] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.774140  8021 tablet_replica.cc:333] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.774716  8021 raft_consensus.cc:2243] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.775555  8021 raft_consensus.cc:2272] T 8a58bd7b2451404a8b70547dee94049a P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.777374  8021 tablet_replica.cc:333] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.777912  8021 raft_consensus.cc:2243] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.778261  8021 raft_consensus.cc:2272] T de7d234ecf1f47fabae92b3be1d2b84d P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.779984  8021 tablet_replica.cc:333] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.780465  8021 raft_consensus.cc:2243] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.781148  8021 raft_consensus.cc:2272] T 1dd94eae6c1241e5838930e00eb43c4c P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.782923  8021 tablet_replica.cc:333] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.783417  8021 raft_consensus.cc:2243] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.783849  8021 raft_consensus.cc:2272] T 62345643f7684ec08f60f740de27cead P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.785619  8021 tablet_replica.cc:333] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.786070  8021 raft_consensus.cc:2243] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.786406  8021 raft_consensus.cc:2272] T 72031807a36e49fa81fe0eb9fb94a7a0 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.788156  8021 tablet_replica.cc:333] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.788645  8021 raft_consensus.cc:2243] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.789300  8021 raft_consensus.cc:2272] T f228d4a700de40ada51433b8043bf6a7 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.790899  8021 tablet_replica.cc:333] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.791376  8021 raft_consensus.cc:2243] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.791714  8021 raft_consensus.cc:2272] T e9db6747820949b6b76dcc3ab05485b1 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.793576  8021 tablet_replica.cc:333] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.794034  8021 raft_consensus.cc:2243] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.794407  8021 raft_consensus.cc:2272] T c66825803ddc46e8849c7718d3cb56ea P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.796285  8021 tablet_replica.cc:333] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.796808  8021 raft_consensus.cc:2243] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.797485  8021 raft_consensus.cc:2272] T a6563b9b2e954646b0ce2535138676d0 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.799151  8021 tablet_replica.cc:333] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.799629  8021 raft_consensus.cc:2243] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.800030  8021 raft_consensus.cc:2272] T 819e82eed43e4968984673887beb39ec P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.802467  8021 tablet_replica.cc:333] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.803217  8021 raft_consensus.cc:2243] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.803840  8021 raft_consensus.cc:2272] T 3699132ec3bb4f8baa0204a17bda51f9 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.807164  8021 tablet_replica.cc:333] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.808168  8021 raft_consensus.cc:2243] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.808830  8021 raft_consensus.cc:2272] T c0f16aeeb02645aea6c44a180eac4f87 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.811955  8021 tablet_replica.cc:333] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.812721  8021 raft_consensus.cc:2243] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.813198  8021 raft_consensus.cc:2272] T 18645784203b4ce1bbe3302513d910c1 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.815653  8021 tablet_replica.cc:333] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.816411  8021 raft_consensus.cc:2243] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.816967  8021 raft_consensus.cc:2272] T e652cb454b674e399c2ac22c22c05506 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.819635  8021 tablet_replica.cc:333] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.820421  8021 raft_consensus.cc:2243] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.820923  8021 raft_consensus.cc:2272] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.823668  8021 tablet_replica.cc:333] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.824432  8021 raft_consensus.cc:2243] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.825544  8021 raft_consensus.cc:2272] T c14505e666fc47d29cfb0c9bc323384e P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.825645 16262 consensus_peers.cc:597] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.828461  8021 tablet_replica.cc:333] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.829246  8021 raft_consensus.cc:2243] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.830155  8021 raft_consensus.cc:2272] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.832803  8021 tablet_replica.cc:333] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.833555  8021 raft_consensus.cc:2243] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.834450  8021 raft_consensus.cc:2272] T 31991d68fa8642189423a50030c4e7d5 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.836587  8021 tablet_replica.cc:333] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.837342  8021 raft_consensus.cc:2243] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:18.837813 16262 consensus_peers.cc:597] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:18.837880  8021 raft_consensus.cc:2272] T b0d1691a17794035a9fae371cbabb4be P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.840127  8021 tablet_replica.cc:333] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.840684  8021 raft_consensus.cc:2243] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.841117  8021 raft_consensus.cc:2272] T 8f5d69137b6e468bb06eb4212b198f84 P 115c2d35eda349178144f8365481e89c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.843575  8021 tablet_replica.cc:333] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.844272  8021 raft_consensus.cc:2243] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.844828  8021 raft_consensus.cc:2272] T b521a909fd464b72bc2742ff3157658b P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.847113  8021 tablet_replica.cc:333] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.847656  8021 raft_consensus.cc:2243] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.848114  8021 raft_consensus.cc:2272] T 26c52cafca664c5f90589901f62e4b16 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.850004  8021 tablet_replica.cc:333] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.850500  8021 raft_consensus.cc:2243] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.851227  8021 raft_consensus.cc:2272] T 2128e5983b684efeacc628400afa861f P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.853063  8021 tablet_replica.cc:333] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.853565  8021 raft_consensus.cc:2243] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.853973  8021 raft_consensus.cc:2272] T ceba3482809245aa87c4e743269df937 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.855669  8021 tablet_replica.cc:333] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.856240  8021 raft_consensus.cc:2243] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.856967  8021 raft_consensus.cc:2272] T 4ba3cb60361b431ba89f7842cdb1c85a P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.858916  8021 tablet_replica.cc:333] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.859429  8021 raft_consensus.cc:2243] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.860452  8021 raft_consensus.cc:2272] T 1dea723251ed4848b42709f873b7d446 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.863122  8021 tablet_replica.cc:333] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.863903  8021 raft_consensus.cc:2243] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.864419  8021 raft_consensus.cc:2272] T 4cec4e308d104a9f9049fa0c2263b58d P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.866358  8021 tablet_replica.cc:333] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.866892  8021 raft_consensus.cc:2243] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.867305  8021 raft_consensus.cc:2272] T 44f529392f494e6d83be5434d71ed0ee P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.869308  8021 tablet_replica.cc:333] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.869800  8021 raft_consensus.cc:2243] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.870198  8021 raft_consensus.cc:2272] T 181b2ad7f8b649d4be93241715d03572 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.872309  8021 tablet_replica.cc:333] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.872992  8021 raft_consensus.cc:2243] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.873536  8021 raft_consensus.cc:2272] T 829bf55d32c344af8ae546eb15903faa P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.875716  8021 tablet_replica.cc:333] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.876278  8021 raft_consensus.cc:2243] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.876699  8021 raft_consensus.cc:2272] T 2201023e16254eed9712e5fba24caa38 P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.878434  8021 tablet_replica.cc:333] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.878914  8021 raft_consensus.cc:2243] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.879637  8021 raft_consensus.cc:2272] T ffb9eaed164d46ccac187f1878f24e4a P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.881552  8021 tablet_replica.cc:333] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.882045  8021 raft_consensus.cc:2243] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.882428  8021 raft_consensus.cc:2272] T 30b15e00d79d42809ddd6a31dd9ee9f3 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.884506  8021 tablet_replica.cc:333] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.885068  8021 raft_consensus.cc:2243] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.885818  8021 raft_consensus.cc:2272] T 7e218598a3114f679b2483143a2f4692 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.887670  8021 tablet_replica.cc:333] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.888196  8021 raft_consensus.cc:2243] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.888577  8021 raft_consensus.cc:2272] T 55b5e6c6090448ab831f8afeef43db0d P 115c2d35eda349178144f8365481e89c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.890374  8021 tablet_replica.cc:333] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.890861  8021 raft_consensus.cc:2243] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:18.891245  8021 raft_consensus.cc:2272] T 424dbaa8a798455b9ae4d2f0b0989561 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:18.893154  8021 tablet_replica.cc:333] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c: stopping tablet replica
I20260709 13:07:18.893600  8021 raft_consensus.cc:2243] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:18.894397  8021 raft_consensus.cc:2272] T 361bd03c04594782911afd84f865c9f5 P 115c2d35eda349178144f8365481e89c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:18.897475 16336 consensus_peers.cc:597] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 -> Peer 3ce6da6d71124256be0679451014f776 (127.7.213.66:43495): Couldn't send request to peer 3ce6da6d71124256be0679451014f776. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:43495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.946012 16337 consensus_peers.cc:597] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Network error: recv got EOF from 127.7.213.67:36993 (error 108). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.946394  8021 connection.cc:466] server connection from 127.0.0.1:37816 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37816 (request call id 716) could send its response
W20260709 13:07:18.958606 16337 consensus_peers.cc:597] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:36993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.970266 16337 consensus_peers.cc:597] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:36993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:18.985536 16337 consensus_peers.cc:597] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:36993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:19.015229 16337 consensus_peers.cc:597] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:36993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:19.018137 16337 consensus_peers.cc:597] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:36993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:19.025134  8021 tablet_server.cc:196] TabletServer@127.7.213.67:0 shutdown complete.
I20260709 13:07:19.091624 17275 raft_consensus.cc:493] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:19.092432 17275 raft_consensus.cc:515] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:19.100812 17275 leader_election.cc:290] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:07:19.103111 16337 consensus_peers.cc:597] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:36993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:19.105624 16337 leader_election.cc:336] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Network error: Client connection negotiation failed: client connection to 127.7.213.67:36993: connect: Connection refused (error 111)
W20260709 13:07:19.107700 16338 leader_election.cc:336] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:19.108122 16338 leader_election.cc:304] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:19.108831 17275 raft_consensus.cc:2749] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:19.119200  8021 tablet_server.cc:179] TabletServer@127.7.213.68:0 shutting down...
I20260709 13:07:19.129601 17275 raft_consensus.cc:493] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aa8eb07867e7420b80a7e615d570509d)
I20260709 13:07:19.131587 17275 raft_consensus.cc:515] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "115c2d35eda349178144f8365481e89c" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36993 } } peers { permanent_uuid: "c8106927ed954e1c800869427b6c7964" member_type: VOTER last_known_addr { host: "127.7.213.68" port: 36791 } } peers { permanent_uuid: "aa8eb07867e7420b80a7e615d570509d" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 40593 } }
I20260709 13:07:19.134091 17275 leader_election.cc:290] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 115c2d35eda349178144f8365481e89c (127.7.213.67:36993), aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593)
W20260709 13:07:19.137475 16337 leader_election.cc:336] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Network error: Client connection negotiation failed: client connection to 127.7.213.67:36993: connect: Connection refused (error 111)
W20260709 13:07:19.144870 16338 leader_election.cc:336] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer aa8eb07867e7420b80a7e615d570509d (127.7.213.65:40593): Network error: Client connection negotiation failed: client connection to 127.7.213.65:40593: connect: Connection refused (error 111)
I20260709 13:07:19.145473 16338 leader_election.cc:304] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8106927ed954e1c800869427b6c7964; no voters: 115c2d35eda349178144f8365481e89c, aa8eb07867e7420b80a7e615d570509d
I20260709 13:07:19.146158 17275 raft_consensus.cc:2749] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:07:19.172469  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:19.173447  8021 tablet_replica.cc:333] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.174149  8021 raft_consensus.cc:2243] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.174808  8021 raft_consensus.cc:2272] T 38bbd6b3bdff467f9d48329073cf0167 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:19.175268 16337 consensus_peers.cc:597] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 -> Peer 115c2d35eda349178144f8365481e89c (127.7.213.67:36993): Couldn't send request to peer 115c2d35eda349178144f8365481e89c. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.67:36993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:19.176990  8021 tablet_replica.cc:333] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.177716  8021 raft_consensus.cc:2243] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.178140  8021 raft_consensus.cc:2272] T 72031807a36e49fa81fe0eb9fb94a7a0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.180454  8021 tablet_replica.cc:333] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.181113  8021 raft_consensus.cc:2243] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.181972  8021 raft_consensus.cc:2272] T 1047f65a57d24cafbf3f41f5b235556e P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.184341  8021 tablet_replica.cc:333] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.184930  8021 raft_consensus.cc:2243] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.185369  8021 raft_consensus.cc:2272] T 09715ac5d7d741e3aabf462dcab633ba P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.187328  8021 tablet_replica.cc:333] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.187896  8021 raft_consensus.cc:2243] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.188320  8021 raft_consensus.cc:2272] T 0255ba6a147b416d8e3081a4e28bf626 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.190253  8021 tablet_replica.cc:333] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.190800  8021 raft_consensus.cc:2243] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.191182  8021 raft_consensus.cc:2272] T de7d234ecf1f47fabae92b3be1d2b84d P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.192909  8021 tablet_replica.cc:333] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.193429  8021 raft_consensus.cc:2243] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.194134  8021 raft_consensus.cc:2272] T 42c540d71b564c08b9032559d7975cc5 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.196056  8021 tablet_replica.cc:333] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.196645  8021 raft_consensus.cc:2243] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.197021  8021 raft_consensus.cc:2272] T b521a909fd464b72bc2742ff3157658b P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.198906  8021 tablet_replica.cc:333] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.199515  8021 raft_consensus.cc:2243] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.200223  8021 raft_consensus.cc:2272] T 7186244ab72e4b32ad068941b913f052 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.202260  8021 tablet_replica.cc:333] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.202804  8021 raft_consensus.cc:2243] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.203264  8021 raft_consensus.cc:2272] T a6563b9b2e954646b0ce2535138676d0 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.205106  8021 tablet_replica.cc:333] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.205660  8021 raft_consensus.cc:2243] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.206073  8021 raft_consensus.cc:2272] T a042ac05958049c38414fa65d0c39414 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.207825  8021 tablet_replica.cc:333] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.208335  8021 raft_consensus.cc:2243] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.208722  8021 raft_consensus.cc:2272] T 18645784203b4ce1bbe3302513d910c1 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.210382  8021 tablet_replica.cc:333] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.210896  8021 raft_consensus.cc:2243] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.211602  8021 raft_consensus.cc:2272] T e652cb454b674e399c2ac22c22c05506 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.213444  8021 tablet_replica.cc:333] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.214434  8021 raft_consensus.cc:2243] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.215049  8021 raft_consensus.cc:2272] T e1e24ed51cb74d0da9f8b77dcbdab5b9 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.218223  8021 tablet_replica.cc:333] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.219162  8021 raft_consensus.cc:2243] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.219676  8021 raft_consensus.cc:2272] T 3c337868c6fc4e079f20a204e451ab66 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.222620  8021 tablet_replica.cc:333] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.223634  8021 raft_consensus.cc:2243] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.224316  8021 raft_consensus.cc:2272] T 31991d68fa8642189423a50030c4e7d5 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.227128  8021 tablet_replica.cc:333] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.227828  8021 raft_consensus.cc:2243] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.229007  8021 raft_consensus.cc:2272] T abb840ca52d04206a82103fb4eb79ccf P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.232023  8021 tablet_replica.cc:333] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.232893  8021 raft_consensus.cc:2243] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.233444  8021 raft_consensus.cc:2272] T 8b5ad93fb43d4467ac4dc1f70ccb0978 P c8106927ed954e1c800869427b6c7964 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.236085  8021 tablet_replica.cc:333] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.236832  8021 raft_consensus.cc:2243] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.237381  8021 raft_consensus.cc:2272] T 7973b570c8554e048bd5a9d02fd13a46 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.240077  8021 tablet_replica.cc:333] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.240933  8021 raft_consensus.cc:2243] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.241473  8021 raft_consensus.cc:2272] T 8a58bd7b2451404a8b70547dee94049a P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.244024  8021 tablet_replica.cc:333] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.244751  8021 raft_consensus.cc:2243] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.245268  8021 raft_consensus.cc:2272] T a6959f2cf9714e91aee31bf53776915b P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.248145  8021 tablet_replica.cc:333] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.248950  8021 raft_consensus.cc:2243] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.249437  8021 raft_consensus.cc:2272] T af66e3a8024e47fe925b7afad68d41c2 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.251199  8021 tablet_replica.cc:333] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.251799  8021 raft_consensus.cc:2243] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.252887  8021 raft_consensus.cc:2272] T c0cbd7d8de354586aab3116154d3ee3c P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.255549  8021 tablet_replica.cc:333] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.256392  8021 raft_consensus.cc:2243] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.256973  8021 raft_consensus.cc:2272] T b90ea5d39bbf4d79a4a44959a8f40359 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.259606  8021 tablet_replica.cc:333] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.260228  8021 raft_consensus.cc:2243] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.260720  8021 raft_consensus.cc:2272] T 2c751e8904a94d8286347a543fbbc90c P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.262494  8021 tablet_replica.cc:333] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.263021  8021 raft_consensus.cc:2243] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.263906  8021 raft_consensus.cc:2272] T 424dbaa8a798455b9ae4d2f0b0989561 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.265859  8021 tablet_replica.cc:333] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.266347  8021 raft_consensus.cc:2243] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.266733  8021 raft_consensus.cc:2272] T 361bd03c04594782911afd84f865c9f5 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.268534  8021 tablet_replica.cc:333] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.269083  8021 raft_consensus.cc:2243] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.269809  8021 raft_consensus.cc:2272] T b0d1691a17794035a9fae371cbabb4be P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.271711  8021 tablet_replica.cc:333] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.272259  8021 raft_consensus.cc:2243] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.272926  8021 raft_consensus.cc:2272] T 181b2ad7f8b649d4be93241715d03572 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.274708  8021 tablet_replica.cc:333] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.275213  8021 raft_consensus.cc:2243] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.275652  8021 raft_consensus.cc:2272] T f603bf0f99d0428194f0998267e8713d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.277601  8021 tablet_replica.cc:333] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.278066  8021 raft_consensus.cc:2243] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.278731  8021 raft_consensus.cc:2272] T bd508336a7c844cc84117b7ef3a565ba P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.280560  8021 tablet_replica.cc:333] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.281044  8021 raft_consensus.cc:2243] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.281418  8021 raft_consensus.cc:2272] T ffb9eaed164d46ccac187f1878f24e4a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.283190  8021 tablet_replica.cc:333] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.283700  8021 raft_consensus.cc:2243] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.284140  8021 raft_consensus.cc:2272] T 819e82eed43e4968984673887beb39ec P c8106927ed954e1c800869427b6c7964 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.285784  8021 tablet_replica.cc:333] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.286260  8021 raft_consensus.cc:2243] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.286959  8021 raft_consensus.cc:2272] T c0f16aeeb02645aea6c44a180eac4f87 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.288722  8021 tablet_replica.cc:333] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.289201  8021 raft_consensus.cc:2243] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.289884  8021 raft_consensus.cc:2272] T 3699132ec3bb4f8baa0204a17bda51f9 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.291656  8021 tablet_replica.cc:333] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.292300  8021 raft_consensus.cc:2243] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.292697  8021 raft_consensus.cc:2272] T 30b15e00d79d42809ddd6a31dd9ee9f3 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.294413  8021 tablet_replica.cc:333] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.294894  8021 raft_consensus.cc:2243] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.295277  8021 raft_consensus.cc:2272] T 7e218598a3114f679b2483143a2f4692 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.297173  8021 tablet_replica.cc:333] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.297683  8021 raft_consensus.cc:2243] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.298385  8021 raft_consensus.cc:2272] T 55b5e6c6090448ab831f8afeef43db0d P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.300251  8021 tablet_replica.cc:333] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.300752  8021 raft_consensus.cc:2243] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.301250  8021 raft_consensus.cc:2272] T ceba3482809245aa87c4e743269df937 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.302987  8021 tablet_replica.cc:333] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.303457  8021 raft_consensus.cc:2243] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.304154  8021 raft_consensus.cc:2272] T 4e9a1ca7eb1348438944c99fcc3eb8e9 P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.305878  8021 tablet_replica.cc:333] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.306325  8021 raft_consensus.cc:2243] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.306700  8021 raft_consensus.cc:2272] T 47e0028829c94c99a91ca4cc4d629a4a P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.308375  8021 tablet_replica.cc:333] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.308832  8021 raft_consensus.cc:2243] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.309208  8021 raft_consensus.cc:2272] T 1dea723251ed4848b42709f873b7d446 P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.310832  8021 tablet_replica.cc:333] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.311259  8021 raft_consensus.cc:2243] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.311978  8021 raft_consensus.cc:2272] T 4cec4e308d104a9f9049fa0c2263b58d P c8106927ed954e1c800869427b6c7964 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.313634  8021 tablet_replica.cc:333] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964: stopping tablet replica
I20260709 13:07:19.314049  8021 raft_consensus.cc:2243] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:19.314430  8021 raft_consensus.cc:2272] T 44f529392f494e6d83be5434d71ed0ee P c8106927ed954e1c800869427b6c7964 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.379868  8021 tablet_server.cc:196] TabletServer@127.7.213.68:0 shutdown complete.
I20260709 13:07:19.446202  8021 master.cc:562] Master@127.7.213.126:46449 shutting down...
W20260709 13:07:19.479571 16096 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:07:19.480743  8021 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:19.481266  8021 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:19.481639  8021 tablet_replica.cc:333] T 00000000000000000000000000000000 P 33af400ee6fd4c1da4b7aa37a0c7b2b0: stopping tablet replica
I20260709 13:07:19.504248  8021 master.cc:584] Master@127.7.213.126:46449 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (33016 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:07:19.595045  8021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.213.126:35643
I20260709 13:07:19.596519  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:19.602231 17291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:19.602728 17292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:19.603706 17294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:19.603874  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:19.605134  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:19.605316  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:19.605523  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602439605489 us; error 0 us; skew 500 ppm
I20260709 13:07:19.606078  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:19.608526  8021 webserver.cc:533] Webserver started at http://127.7.213.126:35407/ using document root <none> and password file <none>
I20260709 13:07:19.609086  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:19.609262  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:19.609549  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:19.610659  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/master-0-root/instance:
uuid: "53474a5fdc5444ccb775be32b9b339c8"
format_stamp: "Formatted at 2026-07-09 13:07:19 on dist-test-slave-9g29"
I20260709 13:07:19.615118  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:07:19.618450 17299 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:19.619310  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:19.619556  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/master-0-root
uuid: "53474a5fdc5444ccb775be32b9b339c8"
format_stamp: "Formatted at 2026-07-09 13:07:19 on dist-test-slave-9g29"
I20260709 13:07:19.619846  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:19.635833  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:19.637077  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:19.677520  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.126:35643
I20260709 13:07:19.677627 17350 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.126:35643 every 8 connection(s)
I20260709 13:07:19.681962 17351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:19.692739 17351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8: Bootstrap starting.
I20260709 13:07:19.697683 17351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:19.702280 17351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8: No bootstrap required, opened a new log
I20260709 13:07:19.704568 17351 raft_consensus.cc:359] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53474a5fdc5444ccb775be32b9b339c8" member_type: VOTER }
I20260709 13:07:19.704964 17351 raft_consensus.cc:385] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:19.705217 17351 raft_consensus.cc:740] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53474a5fdc5444ccb775be32b9b339c8, State: Initialized, Role: FOLLOWER
I20260709 13:07:19.705895 17351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53474a5fdc5444ccb775be32b9b339c8" member_type: VOTER }
I20260709 13:07:19.706422 17351 raft_consensus.cc:399] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:19.706627 17351 raft_consensus.cc:493] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:19.706933 17351 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:19.711562 17351 raft_consensus.cc:515] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53474a5fdc5444ccb775be32b9b339c8" member_type: VOTER }
I20260709 13:07:19.712157 17351 leader_election.cc:304] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53474a5fdc5444ccb775be32b9b339c8; no voters: 
I20260709 13:07:19.713358 17351 leader_election.cc:290] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:19.713949 17354 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:19.715377 17354 raft_consensus.cc:697] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 1 LEADER]: Becoming Leader. State: Replica: 53474a5fdc5444ccb775be32b9b339c8, State: Running, Role: LEADER
I20260709 13:07:19.716084 17354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53474a5fdc5444ccb775be32b9b339c8" member_type: VOTER }
I20260709 13:07:19.716334 17351 sys_catalog.cc:565] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:19.721071 17356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53474a5fdc5444ccb775be32b9b339c8. Latest consensus state: current_term: 1 leader_uuid: "53474a5fdc5444ccb775be32b9b339c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53474a5fdc5444ccb775be32b9b339c8" member_type: VOTER } }
I20260709 13:07:19.721458 17355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53474a5fdc5444ccb775be32b9b339c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53474a5fdc5444ccb775be32b9b339c8" member_type: VOTER } }
I20260709 13:07:19.721931 17356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:19.722375 17355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:19.725996 17362 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:19.733846 17362 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:19.734933  8021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:19.743063 17362 catalog_manager.cc:1383] Generated new cluster ID: 0bee4c0d933d4cbb8077a8fa23a47b1a
I20260709 13:07:19.743420 17362 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:19.762207 17362 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:19.763494 17362 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:19.776849 17362 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8: Generated new TSK 0
I20260709 13:07:19.777535 17362 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:19.802215  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:19.808434 17372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:19.809477 17373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:19.813898  8021 server_base.cc:1061] running on GCE node
W20260709 13:07:19.813921 17375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:19.816131  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:19.816326  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:19.816545  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602439816525 us; error 0 us; skew 500 ppm
I20260709 13:07:19.817080  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:19.819520  8021 webserver.cc:533] Webserver started at http://127.7.213.65:40477/ using document root <none> and password file <none>
I20260709 13:07:19.820009  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:19.820179  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:19.820384  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:19.821405  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-0-root/instance:
uuid: "4ee5229f27334405a8010f6ee263c794"
format_stamp: "Formatted at 2026-07-09 13:07:19 on dist-test-slave-9g29"
I20260709 13:07:19.825938  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:19.829316 17380 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:19.830112  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:07:19.830374  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "4ee5229f27334405a8010f6ee263c794"
format_stamp: "Formatted at 2026-07-09 13:07:19 on dist-test-slave-9g29"
I20260709 13:07:19.830623  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:19.841367  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:19.842520  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:19.843971  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:19.846236  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:19.846408  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:19.846725  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:19.846881  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:19.888000  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:42125
I20260709 13:07:19.888127 17442 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:42125 every 8 connection(s)
I20260709 13:07:19.892889  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:19.900372 17447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:19.901602 17448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:19.904078 17450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:19.904929  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:19.906240  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:19.906445  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:19.907002  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602439906975 us; error 0 us; skew 500 ppm
I20260709 13:07:19.907557  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:19.907792 17443 heartbeater.cc:344] Connected to a master server at 127.7.213.126:35643
I20260709 13:07:19.908241 17443 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:19.909193 17443 heartbeater.cc:507] Master 127.7.213.126:35643 requested a full tablet report, sending...
I20260709 13:07:19.910357  8021 webserver.cc:533] Webserver started at http://127.7.213.66:40415/ using document root <none> and password file <none>
I20260709 13:07:19.910843  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:19.911011  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:19.911291  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:19.911626 17316 ts_manager.cc:194] Registered new tserver with Master: 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125)
I20260709 13:07:19.912465  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-1-root/instance:
uuid: "e1ccf9084e5649ff824aae27144c3fa6"
format_stamp: "Formatted at 2026-07-09 13:07:19 on dist-test-slave-9g29"
I20260709 13:07:19.913486 17316 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46750
I20260709 13:07:19.917491  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:07:19.921428 17455 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:19.922448  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:07:19.922871  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-1-root
uuid: "e1ccf9084e5649ff824aae27144c3fa6"
format_stamp: "Formatted at 2026-07-09 13:07:19 on dist-test-slave-9g29"
I20260709 13:07:19.923167  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:19.949278  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:19.950469  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:19.952042  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:19.954228  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:19.954458  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:19.954695  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:19.954862  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:19.994706  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.66:38535
I20260709 13:07:19.994848 17517 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.66:38535 every 8 connection(s)
I20260709 13:07:19.999861  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:20.007391 17521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:20.007818 17522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:20.011644 17524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:20.011688  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:20.012981  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:20.013222  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:20.013450  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602440013432 us; error 0 us; skew 500 ppm
I20260709 13:07:20.013734 17518 heartbeater.cc:344] Connected to a master server at 127.7.213.126:35643
I20260709 13:07:20.014037  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:20.014139 17518 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:20.015033 17518 heartbeater.cc:507] Master 127.7.213.126:35643 requested a full tablet report, sending...
I20260709 13:07:20.017163  8021 webserver.cc:533] Webserver started at http://127.7.213.67:42873/ using document root <none> and password file <none>
I20260709 13:07:20.017206 17316 ts_manager.cc:194] Registered new tserver with Master: e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:20.017946  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:20.018170  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:20.018502  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:20.019073 17316 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46758
I20260709 13:07:20.019680  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-2-root/instance:
uuid: "de549fcb09b7455eabcbf546a3604373"
format_stamp: "Formatted at 2026-07-09 13:07:20 on dist-test-slave-9g29"
I20260709 13:07:20.024344  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:07:20.028003 17529 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:20.028957  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:07:20.029278  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-2-root
uuid: "de549fcb09b7455eabcbf546a3604373"
format_stamp: "Formatted at 2026-07-09 13:07:20 on dist-test-slave-9g29"
I20260709 13:07:20.029574  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:20.044661  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:20.045886  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:20.047593  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:20.050032  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:20.050294  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:20.050550  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:20.050726  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:20.092078  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.67:39751
I20260709 13:07:20.092221 17591 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.67:39751 every 8 connection(s)
I20260709 13:07:20.106281 17592 heartbeater.cc:344] Connected to a master server at 127.7.213.126:35643
I20260709 13:07:20.106716 17592 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:20.107389 17592 heartbeater.cc:507] Master 127.7.213.126:35643 requested a full tablet report, sending...
I20260709 13:07:20.109302 17316 ts_manager.cc:194] Registered new tserver with Master: de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:20.109977  8021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014752188s
I20260709 13:07:20.111145 17316 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46768
I20260709 13:07:20.136655 17316 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46774:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:20.139614 17316 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:20.211658 17483 tablet_service.cc:1511] Processing CreateTablet for tablet ab5343c89204428f9d6f571036f2f100 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:20.213177 17483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5343c89204428f9d6f571036f2f100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.213092 17482 tablet_service.cc:1511] Processing CreateTablet for tablet 34d273816f6244b78d341ddb9bec525e (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:20.215627 17482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d273816f6244b78d341ddb9bec525e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.223507 17481 tablet_service.cc:1511] Processing CreateTablet for tablet 82ce26f2d71846ea8edacc70b2e04f6b (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:20.225018 17481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ce26f2d71846ea8edacc70b2e04f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.232482 17480 tablet_service.cc:1511] Processing CreateTablet for tablet fda897ab3f4d4e9fac83c26a24e4ac04 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:20.234066 17480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fda897ab3f4d4e9fac83c26a24e4ac04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.238894 17557 tablet_service.cc:1511] Processing CreateTablet for tablet ab5343c89204428f9d6f571036f2f100 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:20.240495 17556 tablet_service.cc:1511] Processing CreateTablet for tablet 34d273816f6244b78d341ddb9bec525e (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:20.241291 17557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5343c89204428f9d6f571036f2f100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.242584 17555 tablet_service.cc:1511] Processing CreateTablet for tablet 82ce26f2d71846ea8edacc70b2e04f6b (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:20.243994 17555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ce26f2d71846ea8edacc70b2e04f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.243571 17479 tablet_service.cc:1511] Processing CreateTablet for tablet 25734691e67748a8955d13a8ef81f786 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:20.245616 17479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25734691e67748a8955d13a8ef81f786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.248879 17554 tablet_service.cc:1511] Processing CreateTablet for tablet fda897ab3f4d4e9fac83c26a24e4ac04 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:20.250336 17554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fda897ab3f4d4e9fac83c26a24e4ac04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.254853 17556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d273816f6244b78d341ddb9bec525e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.255897 17553 tablet_service.cc:1511] Processing CreateTablet for tablet 25734691e67748a8955d13a8ef81f786 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:20.257333 17553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25734691e67748a8955d13a8ef81f786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.260727 17478 tablet_service.cc:1511] Processing CreateTablet for tablet 763080674d2b4bf384e4a5edfccd2ae8 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:20.262826 17478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763080674d2b4bf384e4a5edfccd2ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.298916 17607 tablet_bootstrap.cc:492] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:20.301390 17552 tablet_service.cc:1511] Processing CreateTablet for tablet 763080674d2b4bf384e4a5edfccd2ae8 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:20.302870 17552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763080674d2b4bf384e4a5edfccd2ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.304163 17481 tablet_service.cc:1511] Processing CreateTablet for tablet 590942b106c341d1897bf550b0ff98a6 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:20.305456 17481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590942b106c341d1897bf550b0ff98a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.305549 17607 tablet_bootstrap.cc:654] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.315295 17408 tablet_service.cc:1511] Processing CreateTablet for tablet ab5343c89204428f9d6f571036f2f100 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:20.317229 17408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5343c89204428f9d6f571036f2f100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.321128 17551 tablet_service.cc:1511] Processing CreateTablet for tablet 590942b106c341d1897bf550b0ff98a6 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:20.321323 17406 tablet_service.cc:1511] Processing CreateTablet for tablet 82ce26f2d71846ea8edacc70b2e04f6b (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:20.322557 17551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590942b106c341d1897bf550b0ff98a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.323155 17406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ce26f2d71846ea8edacc70b2e04f6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.325377 17405 tablet_service.cc:1511] Processing CreateTablet for tablet fda897ab3f4d4e9fac83c26a24e4ac04 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:20.327302 17405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fda897ab3f4d4e9fac83c26a24e4ac04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.331192 17404 tablet_service.cc:1511] Processing CreateTablet for tablet 25734691e67748a8955d13a8ef81f786 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:20.333200 17404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25734691e67748a8955d13a8ef81f786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.337318 17403 tablet_service.cc:1511] Processing CreateTablet for tablet 763080674d2b4bf384e4a5edfccd2ae8 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:20.339329 17403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 763080674d2b4bf384e4a5edfccd2ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.341306 17402 tablet_service.cc:1511] Processing CreateTablet for tablet 590942b106c341d1897bf550b0ff98a6 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:20.349565 17607 tablet_bootstrap.cc:492] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:20.350172 17607 ts_tablet_manager.cc:1403] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.052s	user 0.025s	sys 0.015s
I20260709 13:07:20.353319 17607 raft_consensus.cc:359] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.354091 17607 raft_consensus.cc:385] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.354393 17607 raft_consensus.cc:740] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.355237 17607 consensus_queue.cc:260] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.319423 17407 tablet_service.cc:1511] Processing CreateTablet for tablet 34d273816f6244b78d341ddb9bec525e (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:20.359992 17483 tablet_service.cc:1511] Processing CreateTablet for tablet 5d63a3426265488cbfdaf0cc2f5cbbd7 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:20.360338 17553 tablet_service.cc:1511] Processing CreateTablet for tablet 5d63a3426265488cbfdaf0cc2f5cbbd7 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:20.361346 17402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 590942b106c341d1897bf550b0ff98a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.361876 17401 tablet_service.cc:1511] Processing CreateTablet for tablet 5d63a3426265488cbfdaf0cc2f5cbbd7 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:20.364177 17483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d63a3426265488cbfdaf0cc2f5cbbd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.364470 17401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d63a3426265488cbfdaf0cc2f5cbbd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.368299 17407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34d273816f6244b78d341ddb9bec525e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.368669 17553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d63a3426265488cbfdaf0cc2f5cbbd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.380939 17609 tablet_bootstrap.cc:492] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
I20260709 13:07:20.399588 17609 tablet_bootstrap.cc:654] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.418794 17607 ts_tablet_manager.cc:1434] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.068s	user 0.022s	sys 0.026s
I20260709 13:07:20.418933 17609 tablet_bootstrap.cc:492] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:20.419010 17612 tablet_bootstrap.cc:492] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:20.419849 17609 ts_tablet_manager.cc:1403] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.003s
I20260709 13:07:20.419953 17607 tablet_bootstrap.cc:492] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:20.423203 17609 raft_consensus.cc:359] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.424041 17609 raft_consensus.cc:385] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.464129 17400 tablet_service.cc:1511] Processing CreateTablet for tablet 8e57aa54c19f49c491ad03e6f4d42bf9 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:20.471267 17399 tablet_service.cc:1511] Processing CreateTablet for tablet 5e7ecdf2c85e4be188b379c81de5d0b9 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:20.494671 17518 heartbeater.cc:499] Master 127.7.213.126:35643 was elected leader, sending a full tablet report...
I20260709 13:07:20.424429 17609 raft_consensus.cc:740] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
W20260709 13:07:20.502547 17519 tablet.cc:2406] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:20.504323 17553 tablet_service.cc:1511] Processing CreateTablet for tablet 8e57aa54c19f49c491ad03e6f4d42bf9 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:20.505971 17607 tablet_bootstrap.cc:654] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.507004 17556 tablet_service.cc:1511] Processing CreateTablet for tablet 5e7ecdf2c85e4be188b379c81de5d0b9 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:20.508428 17612 tablet_bootstrap.cc:654] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.507020 17553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e57aa54c19f49c491ad03e6f4d42bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.510421 17483 tablet_service.cc:1511] Processing CreateTablet for tablet 5e7ecdf2c85e4be188b379c81de5d0b9 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:20.505759 17609 consensus_queue.cc:260] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.513059 17483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e7ecdf2c85e4be188b379c81de5d0b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.513717 17479 tablet_service.cc:1511] Processing CreateTablet for tablet 8e57aa54c19f49c491ad03e6f4d42bf9 (DEFAULT_TABLE table=test-workload [id=de951d67f62a4618ba6d7bc627f8a4eb]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:20.515488 17479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e57aa54c19f49c491ad03e6f4d42bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.517254 17556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e7ecdf2c85e4be188b379c81de5d0b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.467717 17400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e57aa54c19f49c491ad03e6f4d42bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.523259 17399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e7ecdf2c85e4be188b379c81de5d0b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:20.568346 17609 ts_tablet_manager.cc:1434] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.148s	user 0.010s	sys 0.008s
I20260709 13:07:20.569721 17592 heartbeater.cc:499] Master 127.7.213.126:35643 was elected leader, sending a full tablet report...
I20260709 13:07:20.578089 17607 tablet_bootstrap.cc:492] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:20.578922 17607 ts_tablet_manager.cc:1403] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.159s	user 0.009s	sys 0.009s
I20260709 13:07:20.583039 17609 tablet_bootstrap.cc:492] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
I20260709 13:07:20.582705 17607 raft_consensus.cc:359] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.596156 17607 raft_consensus.cc:385] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.597018 17607 raft_consensus.cc:740] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.597806 17607 consensus_queue.cc:260] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.600188 17607 ts_tablet_manager.cc:1434] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.006s
I20260709 13:07:20.601423 17607 tablet_bootstrap.cc:492] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:20.607458 17612 tablet_bootstrap.cc:492] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:20.608341 17612 ts_tablet_manager.cc:1403] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.190s	user 0.020s	sys 0.005s
I20260709 13:07:20.611872 17612 raft_consensus.cc:359] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.612641 17612 raft_consensus.cc:385] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.612753 17607 tablet_bootstrap.cc:654] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.612990 17612 raft_consensus.cc:740] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.614158 17612 consensus_queue.cc:260] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.617712 17609 tablet_bootstrap.cc:654] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.631831 17443 heartbeater.cc:499] Master 127.7.213.126:35643 was elected leader, sending a full tablet report...
I20260709 13:07:20.632344 17607 tablet_bootstrap.cc:492] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:20.632898 17607 ts_tablet_manager.cc:1403] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.010s
I20260709 13:07:20.636281 17607 raft_consensus.cc:359] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.637002 17607 raft_consensus.cc:385] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.637282 17607 raft_consensus.cc:740] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.638075 17607 consensus_queue.cc:260] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.642488 17612 ts_tablet_manager.cc:1434] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.034s	user 0.015s	sys 0.004s
I20260709 13:07:20.644807 17607 ts_tablet_manager.cc:1434] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.005s
I20260709 13:07:20.646472 17607 tablet_bootstrap.cc:492] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:20.648589 17611 raft_consensus.cc:493] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.649377 17611 raft_consensus.cc:515] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.643640 17612 tablet_bootstrap.cc:492] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:20.657675 17611 leader_election.cc:290] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:20.663852 17612 tablet_bootstrap.cc:654] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.666023 17607 tablet_bootstrap.cc:654] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.668534 17609 tablet_bootstrap.cc:492] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:20.669183 17609 ts_tablet_manager.cc:1403] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.016s
I20260709 13:07:20.684082 17609 raft_consensus.cc:359] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.685106 17609 raft_consensus.cc:385] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.685468 17609 raft_consensus.cc:740] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.693002 17612 tablet_bootstrap.cc:492] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:20.693596 17612 ts_tablet_manager.cc:1403] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.016s
I20260709 13:07:20.695963 17612 raft_consensus.cc:359] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.696708 17612 raft_consensus.cc:385] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.697474 17617 raft_consensus.cc:493] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.698108 17617 raft_consensus.cc:515] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.699064 17609 consensus_queue.cc:260] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.700793 17418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:20.701675 17609 ts_tablet_manager.cc:1434] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:07:20.702891 17609 tablet_bootstrap.cc:492] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
I20260709 13:07:20.704381 17617 leader_election.cc:290] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
W20260709 13:07:20.705250 17457 leader_election.cc:343] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.712009 17607 tablet_bootstrap.cc:492] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:20.697022 17612 raft_consensus.cc:740] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.712749 17607 ts_tablet_manager.cc:1403] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.067s	user 0.030s	sys 0.011s
I20260709 13:07:20.713730 17609 tablet_bootstrap.cc:654] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.714147 17612 consensus_queue.cc:260] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.717267 17607 raft_consensus.cc:359] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:20.720001 17607 raft_consensus.cc:385] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.720325 17607 raft_consensus.cc:740] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.722615 17607 consensus_queue.cc:260] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:20.725250 17612 ts_tablet_manager.cc:1434] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
W20260709 13:07:20.728775 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:20.731164 17612 tablet_bootstrap.cc:492] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:20.732978 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:20.734359 17614 raft_consensus.cc:493] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.734889 17614 raft_consensus.cc:515] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
W20260709 13:07:20.735443 17384 leader_election.cc:343] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.736965 17607 ts_tablet_manager.cc:1434] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:07:20.738173 17607 tablet_bootstrap.cc:492] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:20.747552 17614 leader_election.cc:290] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:20.748698 17607 tablet_bootstrap.cc:654] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.749069 17612 tablet_bootstrap.cc:654] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.763057 17566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373" is_pre_election: true
I20260709 13:07:20.764391 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373" is_pre_election: true
W20260709 13:07:20.767424 17458 leader_election.cc:343] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:20.767321 17383 leader_election.cc:343] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.768021 17458 leader_election.cc:304] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1ccf9084e5649ff824aae27144c3fa6; no voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373
I20260709 13:07:20.768242 17383 leader_election.cc:304] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794; no voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:20.769435 17611 raft_consensus.cc:2749] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:20.769970 17617 raft_consensus.cc:2749] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:20.774991 17614 raft_consensus.cc:493] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.778694 17614 raft_consensus.cc:515] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.780732 17612 tablet_bootstrap.cc:492] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:20.781463 17612 ts_tablet_manager.cc:1403] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.051s	user 0.021s	sys 0.002s
I20260709 13:07:20.781607 17614 leader_election.cc:290] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:20.785669 17612 raft_consensus.cc:359] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.786685 17612 raft_consensus.cc:385] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.787072 17612 raft_consensus.cc:740] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.788440 17612 consensus_queue.cc:260] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.796763 17612 ts_tablet_manager.cc:1434] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.001s
I20260709 13:07:20.798244 17612 tablet_bootstrap.cc:492] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:20.806953 17609 tablet_bootstrap.cc:492] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:20.807544 17609 ts_tablet_manager.cc:1403] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.105s	user 0.040s	sys 0.060s
I20260709 13:07:20.809831 17612 tablet_bootstrap.cc:654] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.811316 17609 raft_consensus.cc:359] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.811693 17418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:20.812122 17609 raft_consensus.cc:385] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.812489 17609 raft_consensus.cc:740] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.812503 17607 tablet_bootstrap.cc:492] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:20.813166 17607 ts_tablet_manager.cc:1403] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.075s	user 0.021s	sys 0.023s
I20260709 13:07:20.813359 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:20.813385 17609 consensus_queue.cc:260] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.816231 17609 ts_tablet_manager.cc:1434] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:20.817165 17609 tablet_bootstrap.cc:492] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
W20260709 13:07:20.817346 17531 leader_election.cc:343] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:20.818408 17531 leader_election.cc:343] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.823443 17607 raft_consensus.cc:359] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.824388 17612 tablet_bootstrap.cc:492] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:20.824918 17609 tablet_bootstrap.cc:654] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.824970 17612 ts_tablet_manager.cc:1403] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.008s
I20260709 13:07:20.824357 17607 raft_consensus.cc:385] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.825543 17607 raft_consensus.cc:740] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.826510 17607 consensus_queue.cc:260] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.828390 17612 raft_consensus.cc:359] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.829140 17612 raft_consensus.cc:385] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.829421 17612 raft_consensus.cc:740] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.830283 17612 consensus_queue.cc:260] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.833739 17609 tablet_bootstrap.cc:492] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:20.834270 17609 ts_tablet_manager.cc:1403] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.001s
I20260709 13:07:20.835804 17612 ts_tablet_manager.cc:1434] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:07:20.836882 17612 tablet_bootstrap.cc:492] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:20.836673 17609 raft_consensus.cc:359] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:20.837342 17609 raft_consensus.cc:385] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.837702 17609 raft_consensus.cc:740] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.838627 17607 ts_tablet_manager.cc:1434] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.003s
I20260709 13:07:20.838531 17609 consensus_queue.cc:260] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:20.840044 17611 raft_consensus.cc:493] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.840550 17611 raft_consensus.cc:515] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.842192 17617 raft_consensus.cc:493] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.842732 17611 leader_election.cc:290] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:20.842690 17617 raft_consensus.cc:515] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.845041 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:20.845916 17609 ts_tablet_manager.cc:1434] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:07:20.846081 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373" is_pre_election: true
I20260709 13:07:20.846745 17567 raft_consensus.cc:2468] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 0.
I20260709 13:07:20.846938 17609 tablet_bootstrap.cc:492] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
W20260709 13:07:20.846609 17457 leader_election.cc:343] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.845079 17617 leader_election.cc:290] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:20.847571 17566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373" is_pre_election: true
I20260709 13:07:20.846593 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:20.849870 17458 leader_election.cc:304] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 4ee5229f27334405a8010f6ee263c794
W20260709 13:07:20.850220 17383 leader_election.cc:343] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:20.851052 17384 leader_election.cc:343] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.851468 17384 leader_election.cc:304] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794; no voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:20.852276 17617 raft_consensus.cc:2749] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:20.852593 17611 raft_consensus.cc:2804] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:20.853039 17611 raft_consensus.cc:493] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.853379 17611 raft_consensus.cc:3060] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:20.854616 17609 tablet_bootstrap.cc:654] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.855693 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:20.856473 17492 raft_consensus.cc:2468] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:20.857875 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:20.858070 17533 leader_election.cc:304] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 4ee5229f27334405a8010f6ee263c794
I20260709 13:07:20.859452 17614 raft_consensus.cc:2804] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:20.859875 17614 raft_consensus.cc:493] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.860232 17614 raft_consensus.cc:3060] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:20.860831 17617 raft_consensus.cc:493] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.861563 17617 raft_consensus.cc:515] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.862067 17611 raft_consensus.cc:515] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.863374 17493 raft_consensus.cc:2393] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de549fcb09b7455eabcbf546a3604373 in current term 1: Already voted for candidate e1ccf9084e5649ff824aae27144c3fa6 in this term.
I20260709 13:07:20.865315 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:20.866469 17617 leader_election.cc:290] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
W20260709 13:07:20.866775 17457 leader_election.cc:343] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.867092 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:20.867729 17492 raft_consensus.cc:2468] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ee5229f27334405a8010f6ee263c794 in term 0.
I20260709 13:07:20.868592 17611 leader_election.cc:290] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:20.869191 17384 leader_election.cc:304] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:20.869498 17612 tablet_bootstrap.cc:654] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.870090 17617 raft_consensus.cc:2804] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:20.870663 17617 raft_consensus.cc:493] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.871162 17617 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:20.871279 17566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:20.871980 17566 raft_consensus.cc:3060] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:20.876298 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373" is_pre_election: true
W20260709 13:07:20.878378 17383 leader_election.cc:343] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.880612 17533 leader_election.cc:304] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de549fcb09b7455eabcbf546a3604373; no voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:20.882688 17612 tablet_bootstrap.cc:492] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:20.883210 17612 ts_tablet_manager.cc:1403] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.008s
I20260709 13:07:20.885341 17614 raft_consensus.cc:515] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.886296 17612 raft_consensus.cc:359] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:20.888434 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794"
W20260709 13:07:20.889979 17531 leader_election.cc:343] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.887040 17612 raft_consensus.cc:385] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.890791 17612 raft_consensus.cc:740] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.891657 17612 consensus_queue.cc:260] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:20.891954 17617 raft_consensus.cc:515] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.895040 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:20.895502 17607 tablet_bootstrap.cc:492] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:20.899855 17493 raft_consensus.cc:3060] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:20.901319 17609 tablet_bootstrap.cc:492] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:20.901247 17617 leader_election.cc:290] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 election: Requested vote from peers e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:20.901964 17609 ts_tablet_manager.cc:1403] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.055s	user 0.031s	sys 0.017s
I20260709 13:07:20.905342 17609 raft_consensus.cc:359] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.905812 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:20.906071 17609 raft_consensus.cc:385] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.906435 17609 raft_consensus.cc:740] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.906490 17492 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:20.907413 17609 consensus_queue.cc:260] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.908635 17612 ts_tablet_manager.cc:1434] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.025s	user 0.011s	sys 0.000s
I20260709 13:07:20.909739 17612 tablet_bootstrap.cc:492] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:20.912217 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:20.897742 17566 raft_consensus.cc:2468] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 1.
I20260709 13:07:20.913506 17636 raft_consensus.cc:2749] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:20.914450 17383 leader_election.cc:343] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:20.915366 17458 leader_election.cc:304] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 4ee5229f27334405a8010f6ee263c794
I20260709 13:07:20.887633 17614 leader_election.cc:290] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:20.916276 17611 raft_consensus.cc:2804] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:20.917943 17609 ts_tablet_manager.cc:1434] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.016s	user 0.014s	sys 0.003s
I20260709 13:07:20.919698 17609 tablet_bootstrap.cc:492] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
I20260709 13:07:20.927862 17612 tablet_bootstrap.cc:654] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.928772 17607 tablet_bootstrap.cc:654] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.921696 17493 raft_consensus.cc:2468] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:20.929498 17611 raft_consensus.cc:697] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:20.930961 17533 leader_election.cc:304] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 4ee5229f27334405a8010f6ee263c794
I20260709 13:07:20.932376 17614 raft_consensus.cc:493] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:20.930825 17611 consensus_queue.cc:237] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.932921 17614 raft_consensus.cc:515] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:20.933502 17638 raft_consensus.cc:2804] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:20.935243 17609 tablet_bootstrap.cc:654] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:20.937229 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5343c89204428f9d6f571036f2f100" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:20.938464 17417 raft_consensus.cc:2468] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:20.940397 17614 leader_election.cc:290] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535), 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125)
I20260709 13:07:20.940485 17531 leader_election.cc:304] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:20.942428 17614 raft_consensus.cc:2804] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:20.942782 17614 raft_consensus.cc:493] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:20.943077 17614 raft_consensus.cc:3060] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:20.929442 17492 raft_consensus.cc:2468] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ee5229f27334405a8010f6ee263c794 in term 1.
I20260709 13:07:20.945142 17384 leader_election.cc:304] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: de549fcb09b7455eabcbf546a3604373
I20260709 13:07:20.946504 17637 raft_consensus.cc:2804] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:20.947027 17637 raft_consensus.cc:697] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 1 LEADER]: Becoming Leader. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Running, Role: LEADER
I20260709 13:07:20.943627 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5343c89204428f9d6f571036f2f100" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:20.947883 17637 consensus_queue.cc:237] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.948201 17493 raft_consensus.cc:2468] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:20.952986 17638 raft_consensus.cc:697] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:20.954674 17614 raft_consensus.cc:515] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:20.953951 17638 consensus_queue.cc:237] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.959076 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5343c89204428f9d6f571036f2f100" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:20.959836 17493 raft_consensus.cc:3060] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:20.972877 17609 tablet_bootstrap.cc:492] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:20.973579 17609 ts_tablet_manager.cc:1403] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.008s
I20260709 13:07:20.972914 17614 leader_election.cc:290] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Requested vote from peers e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535), 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125)
I20260709 13:07:20.977427 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5343c89204428f9d6f571036f2f100" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:20.977759 17493 raft_consensus.cc:2468] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:20.978343 17417 raft_consensus.cc:3060] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:20.983805 17612 tablet_bootstrap.cc:492] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:20.984385 17612 ts_tablet_manager.cc:1403] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.075s	user 0.024s	sys 0.009s
I20260709 13:07:20.985809 17607 tablet_bootstrap.cc:492] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:20.986342 17607 ts_tablet_manager.cc:1403] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.000s
I20260709 13:07:20.987483 17612 raft_consensus.cc:359] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.988348 17612 raft_consensus.cc:385] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.988699 17612 raft_consensus.cc:740] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.988487 17314 catalog_manager.cc:5697] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 0 to 1, leader changed from <none> to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:20.989364 17607 raft_consensus.cc:359] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.989538 17612 consensus_queue.cc:260] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.990181 17607 raft_consensus.cc:385] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:20.990507 17607 raft_consensus.cc:740] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:20.991228 17417 raft_consensus.cc:2468] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:20.992409 17612 ts_tablet_manager.cc:1434] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:20.992686 17531 leader_election.cc:304] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:20.993685 17612 tablet_bootstrap.cc:492] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:20.981200 17316 catalog_manager.cc:5697] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:20.993952 17607 consensus_queue.cc:260] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:20.993635 17614 raft_consensus.cc:2804] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:20.994642 17315 catalog_manager.cc:5697] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ee5229f27334405a8010f6ee263c794 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "4ee5229f27334405a8010f6ee263c794" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:20.996760 17607 ts_tablet_manager.cc:1434] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:07:20.997243 17614 raft_consensus.cc:697] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:20.997912 17607 tablet_bootstrap.cc:492] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:20.998152 17614 consensus_queue.cc:237] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:20.980232 17609 raft_consensus.cc:359] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.000162 17609 raft_consensus.cc:385] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.000525 17609 raft_consensus.cc:740] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.001387 17609 consensus_queue.cc:260] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.001847 17612 tablet_bootstrap.cc:654] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.004024 17609 ts_tablet_manager.cc:1434] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:07:21.005331 17609 tablet_bootstrap.cc:492] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
I20260709 13:07:21.006310 17607 tablet_bootstrap.cc:654] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.021339 17607 tablet_bootstrap.cc:492] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:21.021950 17607 ts_tablet_manager.cc:1403] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.008s
I20260709 13:07:21.023561 17612 tablet_bootstrap.cc:492] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:21.023644 17609 tablet_bootstrap.cc:654] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.024371 17612 ts_tablet_manager.cc:1403] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.012s
I20260709 13:07:21.025570 17614 raft_consensus.cc:493] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.025321 17607 raft_consensus.cc:359] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.026163 17607 raft_consensus.cc:385] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.026111 17614 raft_consensus.cc:515] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.026633 17607 raft_consensus.cc:740] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.027665 17607 consensus_queue.cc:260] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.027897 17612 raft_consensus.cc:359] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.028720 17612 raft_consensus.cc:385] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.029098 17612 raft_consensus.cc:740] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.029654 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:21.029975 17612 consensus_queue.cc:260] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.030339 17417 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.030896 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:21.030282 17614 leader_election.cc:290] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.031518 17493 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.031793 17531 leader_election.cc:304] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:21.033414 17614 raft_consensus.cc:2804] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:21.033586 17607 ts_tablet_manager.cc:1434] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:07:21.033962 17612 ts_tablet_manager.cc:1434] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:07:21.033990 17614 raft_consensus.cc:493] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.034741 17607 tablet_bootstrap.cc:492] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:21.035255 17612 tablet_bootstrap.cc:492] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:21.034773 17614 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.043057 17612 tablet_bootstrap.cc:654] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.045998 17609 tablet_bootstrap.cc:492] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:21.046661 17609 ts_tablet_manager.cc:1403] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.012s
I20260709 13:07:21.049958 17609 raft_consensus.cc:359] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.050944 17609 raft_consensus.cc:385] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.051267 17609 raft_consensus.cc:740] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.052982 17609 consensus_queue.cc:260] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.053359 17607 tablet_bootstrap.cc:654] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.055732 17609 ts_tablet_manager.cc:1434] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:07:21.049822 17614 raft_consensus.cc:515] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.057682 17609 tablet_bootstrap.cc:492] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
I20260709 13:07:21.060320 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:21.061254 17417 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.062543 17614 leader_election.cc:290] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.065496 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:21.066351 17493 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.069134 17314 catalog_manager.cc:5697] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 0 to 1, leader changed from <none> to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:21.071157 17612 tablet_bootstrap.cc:492] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:21.071691 17612 ts_tablet_manager.cc:1403] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.037s	user 0.024s	sys 0.008s
I20260709 13:07:21.074474 17612 raft_consensus.cc:359] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.075162 17612 raft_consensus.cc:385] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.075435 17612 raft_consensus.cc:740] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.076423 17609 tablet_bootstrap.cc:654] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.076195 17612 consensus_queue.cc:260] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.078593 17612 ts_tablet_manager.cc:1434] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:21.079541 17612 tablet_bootstrap.cc:492] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:21.085492 17612 tablet_bootstrap.cc:654] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.090662 17609 tablet_bootstrap.cc:492] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:21.091213 17609 ts_tablet_manager.cc:1403] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.004s
I20260709 13:07:21.093400 17609 raft_consensus.cc:359] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.093916 17609 raft_consensus.cc:385] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.094188 17609 raft_consensus.cc:740] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.094527 17612 tablet_bootstrap.cc:492] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:21.095113 17612 ts_tablet_manager.cc:1403] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.003s
I20260709 13:07:21.095146 17493 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.094983 17609 consensus_queue.cc:260] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.096283 17417 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.096669 17533 leader_election.cc:304] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:21.098138 17612 raft_consensus.cc:359] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.098785 17612 raft_consensus.cc:385] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.099031 17612 raft_consensus.cc:740] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.098237 17614 raft_consensus.cc:2804] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:21.099650 17612 consensus_queue.cc:260] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.100155 17614 raft_consensus.cc:697] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:21.101356 17614 consensus_queue.cc:237] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.102058 17612 ts_tablet_manager.cc:1434] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:07:21.103157 17612 tablet_bootstrap.cc:492] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:21.108635 17638 raft_consensus.cc:493] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.108970 17612 tablet_bootstrap.cc:654] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.109182 17638 raft_consensus.cc:515] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.111572 17638 leader_election.cc:290] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.112130 17609 ts_tablet_manager.cc:1434] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.021s	user 0.000s	sys 0.004s
I20260709 13:07:21.113322 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d63a3426265488cbfdaf0cc2f5cbbd7" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:21.113502 17609 tablet_bootstrap.cc:492] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
I20260709 13:07:21.114109 17417 raft_consensus.cc:2468] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.116814 17531 leader_election.cc:304] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:21.117846 17614 raft_consensus.cc:2804] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:21.118213 17614 raft_consensus.cc:493] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.118597 17614 raft_consensus.cc:3060] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.120046 17612 tablet_bootstrap.cc:492] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: No bootstrap required, opened a new log
I20260709 13:07:21.120229 17607 tablet_bootstrap.cc:492] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:21.120617 17612 ts_tablet_manager.cc:1403] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.006s
I20260709 13:07:21.120739 17607 ts_tablet_manager.cc:1403] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.086s	user 0.021s	sys 0.012s
I20260709 13:07:21.120798 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d63a3426265488cbfdaf0cc2f5cbbd7" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:21.121651 17493 raft_consensus.cc:2468] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.123473 17612 raft_consensus.cc:359] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.124197 17612 raft_consensus.cc:385] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.124444 17612 raft_consensus.cc:740] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.125258 17612 consensus_queue.cc:260] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.126248 17609 tablet_bootstrap.cc:654] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.126132 17614 raft_consensus.cc:515] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.127489 17612 ts_tablet_manager.cc:1434] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 13:07:21.129074 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d63a3426265488cbfdaf0cc2f5cbbd7" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:21.129042 17607 raft_consensus.cc:359] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.129796 17417 raft_consensus.cc:3060] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.129837 17607 raft_consensus.cc:385] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.130282 17607 raft_consensus.cc:740] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.130977 17607 consensus_queue.cc:260] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.133270 17607 ts_tablet_manager.cc:1434] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 13:07:21.134277 17607 tablet_bootstrap.cc:492] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:21.137801 17417 raft_consensus.cc:2468] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.139045 17531 leader_election.cc:304] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:21.139915 17638 raft_consensus.cc:2804] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:21.139986 17607 tablet_bootstrap.cc:654] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.140432 17638 raft_consensus.cc:697] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:21.141324 17638 consensus_queue.cc:237] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.146687 17614 leader_election.cc:290] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.149515 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d63a3426265488cbfdaf0cc2f5cbbd7" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:21.150417 17492 raft_consensus.cc:3060] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.133327 17313 catalog_manager.cc:5697] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 0 to 1, leader changed from <none> to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:21.170797 17492 raft_consensus.cc:2468] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.176256 17607 tablet_bootstrap.cc:492] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:21.179417 17607 ts_tablet_manager.cc:1403] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.032s
I20260709 13:07:21.177229 17609 tablet_bootstrap.cc:492] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:21.180181 17609 ts_tablet_manager.cc:1403] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.067s	user 0.017s	sys 0.040s
I20260709 13:07:21.181632 17607 raft_consensus.cc:359] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.182229 17607 raft_consensus.cc:385] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.182447 17607 raft_consensus.cc:740] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.183074 17607 consensus_queue.cc:260] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.183025 17609 raft_consensus.cc:359] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.184864 17609 raft_consensus.cc:385] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.185111 17609 raft_consensus.cc:740] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.185376 17607 ts_tablet_manager.cc:1434] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:21.186297 17607 tablet_bootstrap.cc:492] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6: Bootstrap starting.
I20260709 13:07:21.185878 17609 consensus_queue.cc:260] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.188694 17609 ts_tablet_manager.cc:1434] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:07:21.188683 17313 catalog_manager.cc:5697] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 0 to 1, leader changed from <none> to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:21.189961 17609 tablet_bootstrap.cc:492] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373: Bootstrap starting.
I20260709 13:07:21.192817 17607 tablet_bootstrap.cc:654] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.196587 17609 tablet_bootstrap.cc:654] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:21.198416 17607 tablet_bootstrap.cc:492] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6: No bootstrap required, opened a new log
I20260709 13:07:21.198861 17611 raft_consensus.cc:493] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.198890 17607 ts_tablet_manager.cc:1403] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.002s
I20260709 13:07:21.199533 17611 raft_consensus.cc:515] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.201951 17609 tablet_bootstrap.cc:492] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373: No bootstrap required, opened a new log
I20260709 13:07:21.202060 17607 raft_consensus.cc:359] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.202553 17609 ts_tablet_manager.cc:1403] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.001s
I20260709 13:07:21.202842 17607 raft_consensus.cc:385] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.203183 17607 raft_consensus.cc:740] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.203269 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7ecdf2c85e4be188b379c81de5d0b9" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:21.203945 17417 raft_consensus.cc:2468] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 0.
I20260709 13:07:21.203905 17607 consensus_queue.cc:260] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.205556 17457 leader_election.cc:304] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:21.203202 17566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7ecdf2c85e4be188b379c81de5d0b9" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373" is_pre_election: true
I20260709 13:07:21.206120 17607 ts_tablet_manager.cc:1434] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:07:21.205660 17609 raft_consensus.cc:359] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.206562 17639 raft_consensus.cc:2804] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:21.206759 17609 raft_consensus.cc:385] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:21.207069 17639 raft_consensus.cc:493] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.207141 17609 raft_consensus.cc:740] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Initialized, Role: FOLLOWER
I20260709 13:07:21.207384 17611 leader_election.cc:290] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:21.207463 17639 raft_consensus.cc:3060] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.208031 17609 consensus_queue.cc:260] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.210901 17566 raft_consensus.cc:2468] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 0.
I20260709 13:07:21.210992 17609 ts_tablet_manager.cc:1434] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:07:21.215500 17637 raft_consensus.cc:493] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.216068 17637 raft_consensus.cc:515] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.217332 17639 raft_consensus.cc:515] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.218180 17637 leader_election.cc:290] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:21.220121 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:21.220062 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7ecdf2c85e4be188b379c81de5d0b9" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:21.220244 17639 leader_election.cc:290] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:21.221060 17417 raft_consensus.cc:3060] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.221457 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7ecdf2c85e4be188b379c81de5d0b9" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:21.221966 17567 raft_consensus.cc:3060] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.219725 17566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373" is_pre_election: true
I20260709 13:07:21.224483 17566 raft_consensus.cc:2393] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4ee5229f27334405a8010f6ee263c794 in current term 1: Already voted for candidate e1ccf9084e5649ff824aae27144c3fa6 in this term.
I20260709 13:07:21.225673 17383 leader_election.cc:304] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794; no voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:21.226462 17637 raft_consensus.cc:3060] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.227420 17417 raft_consensus.cc:2468] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 1.
I20260709 13:07:21.228228 17567 raft_consensus.cc:2468] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 1.
I20260709 13:07:21.228461 17457 leader_election.cc:304] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:21.231942 17639 raft_consensus.cc:2804] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:21.232378 17637 raft_consensus.cc:2749] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:21.232386 17639 raft_consensus.cc:697] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:21.233752 17639 consensus_queue.cc:237] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.241647 17313 catalog_manager.cc:5697] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:21.247390 17638 raft_consensus.cc:493] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.248158 17638 raft_consensus.cc:515] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.250538 17638 leader_election.cc:290] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.251029 17614 raft_consensus.cc:493] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.251621 17614 raft_consensus.cc:515] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.251966 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:21.251452 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:21.252754 17492 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.252887 17417 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.253957 17614 leader_election.cc:290] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.254030 17533 leader_election.cc:304] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:21.254931 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:21.255172 17614 raft_consensus.cc:2804] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:21.255594 17614 raft_consensus.cc:493] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.255690 17417 raft_consensus.cc:2468] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.256065 17614 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.256369 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:21.257038 17492 raft_consensus.cc:2468] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.256922 17531 leader_election.cc:304] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:21.258015 17638 raft_consensus.cc:2804] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:21.258425 17638 raft_consensus.cc:493] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.258808 17638 raft_consensus.cc:3060] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.264407 17614 raft_consensus.cc:515] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.266863 17614 leader_election.cc:290] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.267280 17638 raft_consensus.cc:515] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.267613 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:21.268311 17417 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.268085 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:21.268816 17492 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.270303 17638 leader_election.cc:290] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.270200 17418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:21.270911 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:21.271099 17418 raft_consensus.cc:3060] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.271608 17493 raft_consensus.cc:3060] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.277472 17493 raft_consensus.cc:2468] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.277858 17492 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.278744 17533 leader_election.cc:304] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:21.279601 17638 raft_consensus.cc:2804] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:21.279556 17418 raft_consensus.cc:2468] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.280109 17533 leader_election.cc:304] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:21.280684 17638 raft_consensus.cc:697] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:21.281167 17614 raft_consensus.cc:2804] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:21.281659 17614 raft_consensus.cc:697] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:21.281771 17638 consensus_queue.cc:237] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.282780 17417 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.282485 17614 consensus_queue.cc:237] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.290200 17313 catalog_manager.cc:5697] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 0 to 1, leader changed from <none> to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:21.296253 17637 raft_consensus.cc:493] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.296861 17637 raft_consensus.cc:515] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.298893 17637 leader_election.cc:290] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:21.299549 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:21.300379 17493 raft_consensus.cc:2393] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4ee5229f27334405a8010f6ee263c794 in current term 1: Already voted for candidate de549fcb09b7455eabcbf546a3604373 in this term.
I20260709 13:07:21.300298 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b" candidate_uuid: "4ee5229f27334405a8010f6ee263c794" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373" is_pre_election: true
I20260709 13:07:21.301841 17383 leader_election.cc:304] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794; no voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:21.302552 17637 raft_consensus.cc:3060] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.303287 17313 catalog_manager.cc:5697] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 0 to 1, leader changed from <none> to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:21.308908 17637 raft_consensus.cc:2749] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:21.371454 17614 raft_consensus.cc:493] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:21.372129 17614 raft_consensus.cc:515] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.374436 17614 leader_election.cc:290] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.374938 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e57aa54c19f49c491ad03e6f4d42bf9" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794" is_pre_election: true
I20260709 13:07:21.375110 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e57aa54c19f49c491ad03e6f4d42bf9" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:21.375982 17417 raft_consensus.cc:2468] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.375982 17493 raft_consensus.cc:2468] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 0.
I20260709 13:07:21.377382 17533 leader_election.cc:304] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:21.378165 17614 raft_consensus.cc:2804] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:21.378676 17614 raft_consensus.cc:493] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:21.378986 17614 raft_consensus.cc:3060] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.384341 17614 raft_consensus.cc:515] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.386333 17614 leader_election.cc:290] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:21.387176 17417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e57aa54c19f49c491ad03e6f4d42bf9" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:21.387522 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e57aa54c19f49c491ad03e6f4d42bf9" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:21.387944 17417 raft_consensus.cc:3060] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.388156 17493 raft_consensus.cc:3060] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.390074 17638 consensus_queue.cc:1048] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:21.395644 17493 raft_consensus.cc:2468] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.395689 17417 raft_consensus.cc:2468] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:21.397169 17533 leader_election.cc:304] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:21.398387 17614 raft_consensus.cc:2804] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:21.399083 17614 raft_consensus.cc:697] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:21.400959 17639 consensus_queue.cc:1048] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:21.400380 17614 consensus_queue.cc:237] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:21.404922 17636 consensus_queue.cc:1048] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:21.427208 17639 consensus_queue.cc:1048] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:21.434055 17312 catalog_manager.cc:5697] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 0 to 1, leader changed from <none> to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:21.442943 17566 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:21.454097 17637 consensus_queue.cc:1048] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:21.455916 17637 consensus_queue.cc:1048] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:21.474774  8021 ts_manager.cc:295] Set tserver state for 4ee5229f27334405a8010f6ee263c794 to MAINTENANCE_MODE
I20260709 13:07:21.479422  8021 tablet_server.cc:179] TabletServer@127.7.213.65:0 shutting down...
I20260709 13:07:21.487397 17614 consensus_queue.cc:1048] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:07:21.535794 17531 consensus_peers.cc:597] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.543227  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:21.544416  8021 tablet_replica.cc:333] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.545059  8021 raft_consensus.cc:2243] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.545666  8021 raft_consensus.cc:2272] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.548354  8021 tablet_replica.cc:333] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.548867  8021 raft_consensus.cc:2243] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.549198  8021 raft_consensus.cc:2272] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.551276  8021 tablet_replica.cc:333] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.551853  8021 raft_consensus.cc:2243] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.552176  8021 raft_consensus.cc:2272] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.553972  8021 tablet_replica.cc:333] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.554467  8021 raft_consensus.cc:2243] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.554793  8021 raft_consensus.cc:2272] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.556362  8021 tablet_replica.cc:333] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.556828  8021 raft_consensus.cc:2243] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.557165  8021 raft_consensus.cc:2272] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.559293  8021 tablet_replica.cc:333] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.560046  8021 raft_consensus.cc:2243] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.560842  8021 raft_consensus.cc:2272] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.563292  8021 tablet_replica.cc:333] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.563941  8021 raft_consensus.cc:2243] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.564287  8021 raft_consensus.cc:2272] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.565958  8021 tablet_replica.cc:333] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.566442  8021 raft_consensus.cc:2243] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.566778  8021 raft_consensus.cc:2272] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.568452  8021 tablet_replica.cc:333] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.568881  8021 raft_consensus.cc:2243] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.569203  8021 raft_consensus.cc:2272] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.570699  8021 tablet_replica.cc:333] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: stopping tablet replica
I20260709 13:07:21.571107  8021 raft_consensus.cc:2243] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.571413  8021 raft_consensus.cc:2272] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.600271  8021 tablet_server.cc:196] TabletServer@127.7.213.65:0 shutdown complete.
I20260709 13:07:21.634316 17638 consensus_queue.cc:1048] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:21.648809 17531 consensus_peers.cc:597] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.688398 17614 consensus_queue.cc:1048] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:21.702919 17531 consensus_peers.cc:597] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.707918 17639 consensus_queue.cc:1048] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:21.724215 17457 consensus_peers.cc:597] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.729805 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:21.789019 17614 consensus_queue.cc:1048] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:21.802233 17531 consensus_peers.cc:597] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.806941 17614 consensus_queue.cc:1048] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:21.820590 17531 consensus_peers.cc:597] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.821056 17457 consensus_peers.cc:597] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.863969 17636 consensus_queue.cc:1048] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:07:21.880396 17531 consensus_peers.cc:597] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:21.908424 17531 consensus_peers.cc:597] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:22.730842 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:22.970904 17698 raft_consensus.cc:493] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4ee5229f27334405a8010f6ee263c794)
I20260709 13:07:22.971418 17698 raft_consensus.cc:515] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:22.973117 17698 leader_election.cc:290] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:22.974220 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de549fcb09b7455eabcbf546a3604373" is_pre_election: true
I20260709 13:07:22.975844 17638 raft_consensus.cc:493] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4ee5229f27334405a8010f6ee263c794)
I20260709 13:07:22.976378 17638 raft_consensus.cc:515] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
W20260709 13:07:22.977190 17457 leader_election.cc:336] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111)
I20260709 13:07:22.977680 17457 leader_election.cc:304] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1ccf9084e5649ff824aae27144c3fa6; no voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373
I20260709 13:07:22.978341 17698 raft_consensus.cc:2749] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:22.978432 17638 leader_election.cc:290] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:22.979377 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6" is_pre_election: true
I20260709 13:07:22.979996 17493 raft_consensus.cc:2468] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 1.
I20260709 13:07:22.981069 17533 leader_election.cc:304] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:22.981729 17638 raft_consensus.cc:2804] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:07:22.982028 17531 leader_election.cc:336] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111)
I20260709 13:07:22.982040 17638 raft_consensus.cc:493] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4ee5229f27334405a8010f6ee263c794)
I20260709 13:07:22.982602 17638 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:22.988811 17638 raft_consensus.cc:515] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:22.990483 17638 leader_election.cc:290] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:22.991593 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
W20260709 13:07:22.991825 17531 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111) [suppressed 126 similar messages]
I20260709 13:07:22.992259 17493 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:07:22.994706 17531 leader_election.cc:336] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111)
I20260709 13:07:22.998140 17493 raft_consensus.cc:2468] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 2.
I20260709 13:07:22.999279 17533 leader_election.cc:304] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 4ee5229f27334405a8010f6ee263c794
I20260709 13:07:23.000016 17638 raft_consensus.cc:2804] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:23.000567 17638 raft_consensus.cc:697] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:23.001396 17638 consensus_queue.cc:237] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:23.008383 17313 catalog_manager.cc:5697] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 1 to 2, leader changed from 4ee5229f27334405a8010f6ee263c794 (127.7.213.65) to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:23.501603 17493 raft_consensus.cc:1275] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:23.502815 17638 consensus_queue.cc:1048] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:23.521018 17531 consensus_peers.cc:597] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:23.731891 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:23.898394 17531 consensus_peers.cc:597] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:24.089074 17457 consensus_peers.cc:597] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:24.150100 17531 consensus_peers.cc:597] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:24.224103 17531 consensus_peers.cc:597] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:24.285360 17531 consensus_peers.cc:597] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:24.344571 17531 consensus_peers.cc:597] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:24.348322 17457 consensus_peers.cc:597] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:24.417168 17531 consensus_peers.cc:597] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:24.533440 17531 consensus_peers.cc:597] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:24.732903 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:25.734113 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:07:26.125571 17531 consensus_peers.cc:597] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:07:26.466957 17531 consensus_peers.cc:597] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:26.549062 17531 consensus_peers.cc:597] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:26.619995  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:26.627709 17714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:26.627863 17713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:26.628508 17716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:26.628577  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:26.632109  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:26.632318  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:26.632457  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602446632445 us; error 0 us; skew 500 ppm
I20260709 13:07:26.632925  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:26.635490  8021 webserver.cc:533] Webserver started at http://127.7.213.65:40477/ using document root <none> and password file <none>
I20260709 13:07:26.635982  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:26.636143  8021 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:07:26.639690  8021 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:26.643476 17721 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:26.644291  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:26.644707  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "4ee5229f27334405a8010f6ee263c794"
format_stamp: "Formatted at 2026-07-09 13:07:19 on dist-test-slave-9g29"
I20260709 13:07:26.645063  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:26.664811  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:26.666066  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:26.667554  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:26.671627 17728 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:07:26.672823 17531 consensus_peers.cc:597] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:07:26.716261 17457 consensus_peers.cc:597] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:26.721971  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:07:26.722302  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.052s	user 0.002s	sys 0.002s
I20260709 13:07:26.722555  8021 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:07:26.726948 17728 tablet_bootstrap.cc:492] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
W20260709 13:07:26.734947 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:26.740893 17728 tablet_bootstrap.cc:492] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.741819 17728 tablet_bootstrap.cc:492] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.742511 17728 ts_tablet_manager.cc:1403] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.016s	user 0.004s	sys 0.009s
I20260709 13:07:26.745312 17728 raft_consensus.cc:359] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.746023 17728 raft_consensus.cc:740] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.746773 17728 consensus_queue.cc:260] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.749491 17728 ts_tablet_manager.cc:1434] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.007s
I20260709 13:07:26.750286 17728 tablet_bootstrap.cc:492] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:26.763116 17728 tablet_bootstrap.cc:492] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.763942 17728 tablet_bootstrap.cc:492] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.764520 17728 ts_tablet_manager.cc:1403] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 13:07:26.766501 17728 raft_consensus.cc:359] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.767079 17728 raft_consensus.cc:740] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.767625 17728 consensus_queue.cc:260] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.769124 17728 ts_tablet_manager.cc:1434] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:07:26.769775 17728 tablet_bootstrap.cc:492] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:26.770596  8021 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:07:26.770913  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.048s	user 0.044s	sys 0.001s
W20260709 13:07:26.780797 17531 consensus_peers.cc:597] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:07:26.784137 17728 tablet_bootstrap.cc:492] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.785274 17728 tablet_bootstrap.cc:492] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.786108 17728 ts_tablet_manager.cc:1403] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.004s
I20260709 13:07:26.788916 17728 raft_consensus.cc:359] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.789490 17728 raft_consensus.cc:740] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.790187 17728 consensus_queue.cc:260] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.792116 17728 ts_tablet_manager.cc:1434] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:07:26.792824 17728 tablet_bootstrap.cc:492] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:26.804491 17728 tablet_bootstrap.cc:492] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.805250 17728 tablet_bootstrap.cc:492] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.805845 17728 ts_tablet_manager.cc:1403] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:07:26.808013 17728 raft_consensus.cc:359] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:26.808704 17728 raft_consensus.cc:740] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.809371 17728 consensus_queue.cc:260] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } } peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } }
I20260709 13:07:26.810868 17728 ts_tablet_manager.cc:1434] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:26.811561 17728 tablet_bootstrap.cc:492] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:26.823614 17728 tablet_bootstrap.cc:492] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.824625 17728 tablet_bootstrap.cc:492] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.825220 17728 ts_tablet_manager.cc:1403] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 13:07:26.827152 17728 raft_consensus.cc:359] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.827867 17728 raft_consensus.cc:740] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.828519 17728 consensus_queue.cc:260] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.829993  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:42125
I20260709 13:07:26.830163 17790 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:42125 every 8 connection(s)
I20260709 13:07:26.830307 17728 ts_tablet_manager.cc:1434] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:26.831310 17728 tablet_bootstrap.cc:492] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:26.837179  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:26.845264 17728 tablet_bootstrap.cc:492] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.846226 17728 tablet_bootstrap.cc:492] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.846848 17728 ts_tablet_manager.cc:1403] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:07:26.850091 17728 raft_consensus.cc:359] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.850729 17728 raft_consensus.cc:740] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.851346 17728 consensus_queue.cc:260] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.852792 17728 ts_tablet_manager.cc:1434] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:07:26.853399 17728 tablet_bootstrap.cc:492] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:26.863581 17791 heartbeater.cc:344] Connected to a master server at 127.7.213.126:35643
I20260709 13:07:26.864084 17791 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:26.864969 17791 heartbeater.cc:507] Master 127.7.213.126:35643 requested a full tablet report, sending...
I20260709 13:07:26.865175 17728 tablet_bootstrap.cc:492] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.866425 17728 tablet_bootstrap.cc:492] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.867136 17728 ts_tablet_manager.cc:1403] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:07:26.869270 17567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25734691e67748a8955d13a8ef81f786"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
mode: GRACEFUL
new_leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:36888
I20260709 13:07:26.869396 17728 raft_consensus.cc:359] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.870033 17567 raft_consensus.cc:606] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Received request to transfer leadership to TS e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:26.870175 17728 raft_consensus.cc:740] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.870905 17728 consensus_queue.cc:260] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.872581 17313 ts_manager.cc:194] Re-registered known tserver with Master: 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125)
I20260709 13:07:26.872889 17728 ts_tablet_manager.cc:1434] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:07:26.873754 17728 tablet_bootstrap.cc:492] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:26.873585 17567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
mode: GRACEFUL
new_leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:36888
I20260709 13:07:26.874282 17567 raft_consensus.cc:606] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Received request to transfer leadership to TS e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:26.876902 17567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
mode: GRACEFUL
new_leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:36888
I20260709 13:07:26.877471 17567 raft_consensus.cc:606] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Received request to transfer leadership to TS e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:26.880110 17567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
mode: GRACEFUL
new_leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:36888
I20260709 13:07:26.880661 17567 raft_consensus.cc:606] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Received request to transfer leadership to TS e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:26.881762  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:07:26.882547  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:26.883323  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:26.883594 17313 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35112
I20260709 13:07:26.887271 17791 heartbeater.cc:499] Master 127.7.213.126:35643 was elected leader, sending a full tablet report...
I20260709 13:07:26.888046 17728 tablet_bootstrap.cc:492] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.888906 17728 tablet_bootstrap.cc:492] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.889537 17728 ts_tablet_manager.cc:1403] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:07:26.891892 17728 raft_consensus.cc:359] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.892529 17728 raft_consensus.cc:740] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.893186 17728 consensus_queue.cc:260] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.894515 17728 ts_tablet_manager.cc:1434] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:26.895129 17728 tablet_bootstrap.cc:492] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:26.909010 17728 tablet_bootstrap.cc:492] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.909827 17728 tablet_bootstrap.cc:492] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.910512 17728 ts_tablet_manager.cc:1403] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:07:26.912979 17728 raft_consensus.cc:359] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.913654 17728 raft_consensus.cc:740] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.914450 17728 consensus_queue.cc:260] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.916287 17728 ts_tablet_manager.cc:1434] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:07:26.917285 17728 tablet_bootstrap.cc:492] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: Bootstrap starting.
I20260709 13:07:26.933048 17728 tablet_bootstrap.cc:492] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:26.933738 17728 tablet_bootstrap.cc:492] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: Bootstrap complete.
I20260709 13:07:26.934311 17728 ts_tablet_manager.cc:1403] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:07:26.936110 17728 raft_consensus.cc:359] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.936581 17728 raft_consensus.cc:740] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee5229f27334405a8010f6ee263c794, State: Initialized, Role: FOLLOWER
I20260709 13:07:26.937155 17728 consensus_queue.cc:260] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:26.938902 17728 ts_tablet_manager.cc:1434] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:07:27.018641 17712 raft_consensus.cc:993] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373: : Instructing follower e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:27.019061 17805 raft_consensus.cc:1081] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Signalling peer e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:27.020633 17492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:59808
I20260709 13:07:27.021193 17492 raft_consensus.cc:493] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.021520 17492 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.026825 17492 raft_consensus.cc:515] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.028545 17492 leader_election.cc:290] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:27.029456 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:27.029759 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:27.030161 17765 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.030339 17567 raft_consensus.cc:3055] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:27.030733 17567 raft_consensus.cc:740] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:27.031576 17567 consensus_queue.cc:260] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.032903 17567 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.037853 17765 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:27.039124 17457 leader_election.cc:304] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:27.039819 17567 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:27.039893 17698 raft_consensus.cc:2804] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:27.041317 17698 raft_consensus.cc:697] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:27.042019 17698 consensus_queue.cc:237] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.049371 17313 catalog_manager.cc:5697] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 1 to 2, leader changed from de549fcb09b7455eabcbf546a3604373 (127.7.213.67) to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:27.062008 17765 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.103219 17712 raft_consensus.cc:993] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373: : Instructing follower e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:27.103794 17805 raft_consensus.cc:1081] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Signalling peer e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:27.105211 17492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:59808
I20260709 13:07:27.105684 17492 raft_consensus.cc:493] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.105983 17492 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.112234 17492 raft_consensus.cc:515] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.113850 17492 leader_election.cc:290] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 3 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:27.114559 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:27.114912 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:27.115259 17765 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.115535 17567 raft_consensus.cc:3055] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:27.115875 17567 raft_consensus.cc:740] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:27.116590 17567 consensus_queue.cc:260] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.117487 17567 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.120471 17765 raft_consensus.cc:2468] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 3.
I20260709 13:07:27.121662 17457 leader_election.cc:304] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:27.122341 17698 raft_consensus.cc:2804] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:27.122625 17567 raft_consensus.cc:2468] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 3.
I20260709 13:07:27.122731 17698 raft_consensus.cc:697] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:27.123776 17698 consensus_queue.cc:237] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.129197 17712 raft_consensus.cc:993] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373: : Instructing follower e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:27.129676 17805 raft_consensus.cc:1081] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Signalling peer e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:27.131207 17492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25734691e67748a8955d13a8ef81f786"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:59808
I20260709 13:07:27.131723 17492 raft_consensus.cc:493] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.132164 17492 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.131727 17313 catalog_manager.cc:5697] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 2 to 3, leader changed from de549fcb09b7455eabcbf546a3604373 (127.7.213.67) to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:27.137518 17492 raft_consensus.cc:515] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.139611 17492 leader_election.cc:290] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:27.140218 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:27.140834 17765 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.140826 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:27.141575 17567 raft_consensus.cc:3055] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:27.141889 17567 raft_consensus.cc:740] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:27.142714 17567 consensus_queue.cc:260] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.144066 17567 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.148444 17765 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:27.149631 17457 leader_election.cc:304] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:27.150316 17698 raft_consensus.cc:2804] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:27.150811 17698 raft_consensus.cc:697] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:27.151192 17567 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:27.151674 17698 consensus_queue.cc:237] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.160676 17316 catalog_manager.cc:5697] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 1 to 2, leader changed from de549fcb09b7455eabcbf546a3604373 (127.7.213.67) to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:27.329442 17712 raft_consensus.cc:993] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373: : Instructing follower e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:27.329878 17805 raft_consensus.cc:1081] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Signalling peer e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:27.331235 17492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:59808
I20260709 13:07:27.331789 17492 raft_consensus.cc:493] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.332077 17492 raft_consensus.cc:3060] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.336860 17492 raft_consensus.cc:515] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.338449 17492 leader_election.cc:290] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:27.339273 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:27.339521 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:27.339891 17765 raft_consensus.cc:3060] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.340270 17567 raft_consensus.cc:3055] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:27.340641 17567 raft_consensus.cc:740] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:27.341449 17567 consensus_queue.cc:260] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.342712 17567 raft_consensus.cc:3060] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:27.346408 17765 raft_consensus.cc:2468] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:27.347471 17457 leader_election.cc:304] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:27.347987 17567 raft_consensus.cc:2468] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:27.348232 17698 raft_consensus.cc:2804] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:27.348767 17698 raft_consensus.cc:697] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:27.349555 17698 consensus_queue.cc:237] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.355854 17316 catalog_manager.cc:5697] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 1 to 2, leader changed from de549fcb09b7455eabcbf546a3604373 (127.7.213.67) to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:27.502197 17765 raft_consensus.cc:1275] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.503489 17698 consensus_queue.cc:1048] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:27.513913 17567 raft_consensus.cc:1275] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.515653 17818 consensus_queue.cc:1048] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:27.525091 17765 raft_consensus.cc:1275] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:27.526517 17818 consensus_queue.cc:1048] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:27.538785 17567 raft_consensus.cc:1275] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:27.540459 17818 consensus_queue.cc:1048] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:27.635897 17765 raft_consensus.cc:1275] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.637152 17818 consensus_queue.cc:1048] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:27.647244 17567 raft_consensus.cc:1275] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.649369 17698 consensus_queue.cc:1048] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:27.735931 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:27.893651  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:27.911432 17765 raft_consensus.cc:1275] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.912776 17492 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25734691e67748a8955d13a8ef81f786"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
mode: GRACEFUL
new_leader_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:43798
I20260709 13:07:27.913056 17806 consensus_queue.cc:1048] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:27.913513 17492 raft_consensus.cc:606] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Received request to transfer leadership to TS de549fcb09b7455eabcbf546a3604373
I20260709 13:07:27.916266 17492 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
mode: GRACEFUL
new_leader_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:43798
I20260709 13:07:27.916865 17492 raft_consensus.cc:606] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Received request to transfer leadership to TS de549fcb09b7455eabcbf546a3604373
I20260709 13:07:27.918198  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:27.919095  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:27.920073  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:27.927249 17567 raft_consensus.cc:1275] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:27.928772 17806 consensus_queue.cc:1048] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:27.982281 17818 raft_consensus.cc:993] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6: : Instructing follower de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:27.982784 17806 raft_consensus.cc:1081] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Signalling peer de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:27.984416 17567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:36416
I20260709 13:07:27.985051 17567 raft_consensus.cc:493] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.985394 17567 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.990724 17567 raft_consensus.cc:515] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.992659 17567 leader_election.cc:290] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 3 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:27.993408 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:27.993786 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:27.994104 17765 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.994371 17492 raft_consensus.cc:3055] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:27.994697 17492 raft_consensus.cc:740] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:27.995428 17492 consensus_queue.cc:260] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:27.996634 17492 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.001230 17765 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 3.
I20260709 13:07:28.002599 17531 leader_election.cc:304] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:28.003347 17820 raft_consensus.cc:2804] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:28.004098 17820 raft_consensus.cc:697] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:28.003304 17492 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 3.
I20260709 13:07:28.004945 17820 consensus_queue.cc:237] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:28.011453 17313 catalog_manager.cc:5697] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 2 to 3, leader changed from e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66) to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 3 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.130499 17818 raft_consensus.cc:993] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6: : Instructing follower de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:28.131053 17806 raft_consensus.cc:1081] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Signalling peer de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:28.132858 17567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25734691e67748a8955d13a8ef81f786"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:36416
I20260709 13:07:28.133396 17567 raft_consensus.cc:493] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:28.133813 17567 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.140532 17567 raft_consensus.cc:515] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:28.142297 17567 leader_election.cc:290] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 3 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:28.143103 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:28.143285 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:28.143847 17765 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.143901 17492 raft_consensus.cc:3055] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:28.144375 17492 raft_consensus.cc:740] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:28.145044 17492 consensus_queue.cc:260] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:28.145915 17492 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.149659 17765 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 3.
I20260709 13:07:28.150391 17492 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 3.
I20260709 13:07:28.151005 17531 leader_election.cc:304] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:28.151808 17820 raft_consensus.cc:2804] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:28.152343 17820 raft_consensus.cc:697] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:28.153294 17820 consensus_queue.cc:237] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:28.160072 17313 catalog_manager.cc:5697] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 2 to 3, leader changed from e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66) to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 3 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:28.498327 17765 raft_consensus.cc:1275] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.499583 17820 consensus_queue.cc:1048] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.505568 17492 raft_consensus.cc:1275] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.507207 17820 consensus_queue.cc:1048] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.561815 17492 raft_consensus.cc:1275] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.563021 17820 consensus_queue.cc:1048] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.571440 17765 raft_consensus.cc:1275] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.573678 17839 consensus_queue.cc:1048] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:07:28.736862 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:28.931847  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:28.939668 17492 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
mode: GRACEFUL
new_leader_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:43798
I20260709 13:07:28.940384 17492 raft_consensus.cc:606] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 LEADER]: Received request to transfer leadership to TS de549fcb09b7455eabcbf546a3604373
I20260709 13:07:28.942984 17567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
mode: GRACEFUL
new_leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:36888
I20260709 13:07:28.943502 17567 raft_consensus.cc:606] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Received request to transfer leadership to TS e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:28.946252 17567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e57aa54c19f49c491ad03e6f4d42bf9"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
mode: GRACEFUL
new_leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:36888
I20260709 13:07:28.946924 17567 raft_consensus.cc:606] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Received request to transfer leadership to TS e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:28.948129  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:28.948607  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:28.949124  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:29.037994 17712 raft_consensus.cc:993] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373: : Instructing follower e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:29.038463 17839 raft_consensus.cc:1081] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Signalling peer e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:29.039942 17492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e57aa54c19f49c491ad03e6f4d42bf9"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:59808
I20260709 13:07:29.040419 17492 raft_consensus.cc:493] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:29.040742 17492 raft_consensus.cc:3060] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.045460 17492 raft_consensus.cc:515] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:29.046952 17492 leader_election.cc:290] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:29.048029 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e57aa54c19f49c491ad03e6f4d42bf9" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:29.048029 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e57aa54c19f49c491ad03e6f4d42bf9" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:29.048904 17567 raft_consensus.cc:3055] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:29.048929 17765 raft_consensus.cc:3060] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.049392 17567 raft_consensus.cc:740] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:29.050258 17567 consensus_queue.cc:260] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:29.051498 17567 raft_consensus.cc:3060] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.056387 17765 raft_consensus.cc:2468] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:29.057632 17457 leader_election.cc:304] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:29.058041 17567 raft_consensus.cc:2468] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:29.058293 17806 raft_consensus.cc:2804] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:29.059361 17806 raft_consensus.cc:697] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:29.060194 17806 consensus_queue.cc:237] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:29.066998 17316 catalog_manager.cc:5697] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 1 to 2, leader changed from de549fcb09b7455eabcbf546a3604373 (127.7.213.67) to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.082764 17840 raft_consensus.cc:993] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6: : Instructing follower de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:29.083274 17806 raft_consensus.cc:1081] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 LEADER]: Signalling peer de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:29.084800 17567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:36416
I20260709 13:07:29.085309 17567 raft_consensus.cc:493] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:29.085582 17567 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.089983 17567 raft_consensus.cc:515] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:29.091367 17567 leader_election.cc:290] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 4 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:29.092366 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:29.092422 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "590942b106c341d1897bf550b0ff98a6" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:29.093055 17765 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.093276 17492 raft_consensus.cc:3055] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:29.093607 17492 raft_consensus.cc:740] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:29.094194 17492 consensus_queue.cc:260] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:29.095046 17492 raft_consensus.cc:3060] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:29.098429 17765 raft_consensus.cc:2468] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 4.
I20260709 13:07:29.099339 17492 raft_consensus.cc:2468] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 4.
I20260709 13:07:29.099570 17531 leader_election.cc:304] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:29.100418 17839 raft_consensus.cc:2804] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:29.100805 17839 raft_consensus.cc:697] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 4 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:29.101426 17839 consensus_queue.cc:237] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:29.108392 17316 catalog_manager.cc:5697] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 3 to 4, leader changed from e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66) to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 4 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:29.152978 17820 raft_consensus.cc:993] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373: : Instructing follower e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:29.153426 17820 raft_consensus.cc:1081] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Signalling peer e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:29.154793 17492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:59808
I20260709 13:07:29.155237 17492 raft_consensus.cc:493] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:29.155511 17492 raft_consensus.cc:3060] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.159886 17492 raft_consensus.cc:515] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:29.161254 17492 leader_election.cc:290] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:29.162274 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:29.162458 17567 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ce26f2d71846ea8edacc70b2e04f6b" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:29.162955 17765 raft_consensus.cc:3060] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.163177 17567 raft_consensus.cc:3055] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:29.163582 17567 raft_consensus.cc:740] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:29.164337 17567 consensus_queue.cc:260] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:29.165457 17567 raft_consensus.cc:3060] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:29.168887 17765 raft_consensus.cc:2468] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:29.170104 17457 leader_election.cc:304] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:29.170150 17567 raft_consensus.cc:2468] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 2.
I20260709 13:07:29.170843 17806 raft_consensus.cc:2804] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:29.171257 17806 raft_consensus.cc:697] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:29.172036 17806 consensus_queue.cc:237] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:29.178237 17316 catalog_manager.cc:5697] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 1 to 2, leader changed from de549fcb09b7455eabcbf546a3604373 (127.7.213.67) to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:29.460306 17764 raft_consensus.cc:1275] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:29.461633 17840 consensus_queue.cc:1048] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:29.473750 17567 raft_consensus.cc:1275] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:29.475692 17850 consensus_queue.cc:1048] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:29.508440 17492 raft_consensus.cc:1275] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:29.509600 17820 consensus_queue.cc:1048] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:29.518061 17764 raft_consensus.cc:1275] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:29.520345 17839 consensus_queue.cc:1048] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:29.557605 17567 raft_consensus.cc:1275] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:29.558799 17850 consensus_queue.cc:1048] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:29.567010 17764 raft_consensus.cc:1275] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:29.568405 17850 consensus_queue.cc:1048] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:29.737874 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:29.956373  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:29.963826 17567 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25734691e67748a8955d13a8ef81f786"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
mode: GRACEFUL
new_leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:36888
I20260709 13:07:29.964330 17567 raft_consensus.cc:606] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Received request to transfer leadership to TS e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:29.966805 17492 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e7ecdf2c85e4be188b379c81de5d0b9"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
mode: GRACEFUL
new_leader_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:43798
I20260709 13:07:29.967372 17492 raft_consensus.cc:606] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Received request to transfer leadership to TS de549fcb09b7455eabcbf546a3604373
I20260709 13:07:29.969977 17492 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
mode: GRACEFUL
new_leader_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:43798
I20260709 13:07:29.970479 17492 raft_consensus.cc:606] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Received request to transfer leadership to TS de549fcb09b7455eabcbf546a3604373
I20260709 13:07:29.971596  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:29.972183  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:29.972747  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:30.382616 17857 raft_consensus.cc:993] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6: : Instructing follower de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:30.383236 17850 raft_consensus.cc:1081] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Signalling peer de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:30.383894 17806 raft_consensus.cc:993] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6: : Instructing follower de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:30.384431 17850 raft_consensus.cc:1081] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Signalling peer de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:30.385023 17566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e7ecdf2c85e4be188b379c81de5d0b9"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:36416
I20260709 13:07:30.385748 17566 raft_consensus.cc:493] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:30.386103 17567 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:36416
I20260709 13:07:30.386106 17566 raft_consensus.cc:3060] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.386643 17567 raft_consensus.cc:493] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:30.386987 17567 raft_consensus.cc:3060] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.391855 17567 raft_consensus.cc:515] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:30.392201 17566 raft_consensus.cc:515] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:30.393800 17567 leader_election.cc:290] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:30.394274 17764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:30.394485 17566 leader_election.cc:290] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:30.394641 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:30.395615 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7ecdf2c85e4be188b379c81de5d0b9" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:30.395876 17493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e7ecdf2c85e4be188b379c81de5d0b9" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:30.396063 17492 raft_consensus.cc:3055] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:30.396293 17765 raft_consensus.cc:3060] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.396528 17492 raft_consensus.cc:740] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:30.396555 17493 raft_consensus.cc:3055] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:30.395661 17764 raft_consensus.cc:3060] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.397079 17493 raft_consensus.cc:740] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:30.397333 17492 consensus_queue.cc:260] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:30.398583 17492 raft_consensus.cc:3060] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.398236 17493 consensus_queue.cc:260] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:30.399674 17493 raft_consensus.cc:3060] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:30.402837 17765 raft_consensus.cc:2468] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 2.
I20260709 13:07:30.404872 17492 raft_consensus.cc:2468] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 2.
I20260709 13:07:30.405525 17764 raft_consensus.cc:2468] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 2.
I20260709 13:07:30.405712 17531 leader_election.cc:304] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:30.406389 17533 leader_election.cc:304] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de549fcb09b7455eabcbf546a3604373, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:30.406775 17839 raft_consensus.cc:2804] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:30.407240 17712 raft_consensus.cc:2804] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:30.407408 17839 raft_consensus.cc:697] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:30.407428 17493 raft_consensus.cc:2468] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 2.
I20260709 13:07:30.407811 17712 raft_consensus.cc:697] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:30.408598 17712 consensus_queue.cc:237] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:30.408574 17839 consensus_queue.cc:237] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:30.417047 17312 catalog_manager.cc:5697] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 1 to 2, leader changed from e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66) to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.418455 17712 raft_consensus.cc:993] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373: : Instructing follower e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:30.419015 17839 raft_consensus.cc:1081] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Signalling peer e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:30.420720 17493 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25734691e67748a8955d13a8ef81f786"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:59808
I20260709 13:07:30.421300 17493 raft_consensus.cc:493] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:30.421693 17493 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.429087 17493 raft_consensus.cc:515] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:30.431463 17493 leader_election.cc:290] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 4 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:30.431699 17764 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:30.432197 17566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25734691e67748a8955d13a8ef81f786" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:30.432405 17764 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.432853 17566 raft_consensus.cc:3055] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:30.433120 17566 raft_consensus.cc:740] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:30.433905 17566 consensus_queue.cc:260] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:30.435209 17566 raft_consensus.cc:3060] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:30.436293 17312 catalog_manager.cc:5697] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 1 to 2, leader changed from e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66) to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:30.439002 17764 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 4.
I20260709 13:07:30.440419 17457 leader_election.cc:304] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:30.441278 17850 raft_consensus.cc:2804] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:30.441728 17850 raft_consensus.cc:697] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 4 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:30.442526 17850 consensus_queue.cc:237] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:30.444972 17566 raft_consensus.cc:2468] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 4.
I20260709 13:07:30.451135 17316 catalog_manager.cc:5697] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 3 to 4, leader changed from de549fcb09b7455eabcbf546a3604373 (127.7.213.67) to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:30.739145 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:30.852775 17764 raft_consensus.cc:1275] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.854523 17839 consensus_queue.cc:1048] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:30.868777 17493 raft_consensus.cc:1275] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.870759 17712 consensus_queue.cc:1048] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:30.921880 17764 raft_consensus.cc:1275] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.923988 17712 consensus_queue.cc:1048] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:30.934314 17566 raft_consensus.cc:1275] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:30.935202 17493 raft_consensus.cc:1275] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:30.936753 17857 consensus_queue.cc:1048] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:30.937089 17712 consensus_queue.cc:1048] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:30.960376 17765 raft_consensus.cc:1275] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:30.961688 17857 consensus_queue.cc:1048] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:30.978916  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:30.988190 17566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
mode: GRACEFUL
new_leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:36888
I20260709 13:07:30.988909 17566 raft_consensus.cc:606] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Received request to transfer leadership to TS e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:30.992012 17493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
mode: GRACEFUL
new_leader_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:43798
I20260709 13:07:30.992555 17493 raft_consensus.cc:606] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Received request to transfer leadership to TS de549fcb09b7455eabcbf546a3604373
I20260709 13:07:30.995151 17566 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
mode: GRACEFUL
new_leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:36888
I20260709 13:07:30.995819 17566 raft_consensus.cc:606] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Received request to transfer leadership to TS e1ccf9084e5649ff824aae27144c3fa6
I20260709 13:07:30.996942  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:07:30.997424  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:30.998318  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:31.159055 17839 raft_consensus.cc:993] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373: : Instructing follower e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:31.159591 17712 raft_consensus.cc:1081] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Signalling peer e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:31.161171 17492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:59808
I20260709 13:07:31.161650 17492 raft_consensus.cc:493] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.161914 17492 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.168996 17492 raft_consensus.cc:515] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:31.170550 17492 leader_election.cc:290] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 4 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:31.171516 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:31.172030 17566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34d273816f6244b78d341ddb9bec525e" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:31.172322 17765 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.172674 17566 raft_consensus.cc:3055] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:31.173023 17566 raft_consensus.cc:740] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:31.174254 17566 consensus_queue.cc:260] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:31.176064 17566 raft_consensus.cc:3060] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.181766 17765 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 4.
I20260709 13:07:31.183187 17457 leader_election.cc:304] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:31.183996 17857 raft_consensus.cc:2804] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:31.184417 17857 raft_consensus.cc:697] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 4 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:31.184834 17566 raft_consensus.cc:2468] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 4.
I20260709 13:07:31.185635 17857 consensus_queue.cc:237] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:31.193928 17313 catalog_manager.cc:5697] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 3 to 4, leader changed from de549fcb09b7455eabcbf546a3604373 (127.7.213.67) to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 4 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:31.285288 17850 raft_consensus.cc:993] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6: : Instructing follower de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:31.285753 17857 raft_consensus.cc:1081] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Signalling peer de549fcb09b7455eabcbf546a3604373 to start an election
I20260709 13:07:31.287287 17566 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8"
dest_uuid: "de549fcb09b7455eabcbf546a3604373"
 from {username='slave'} at 127.0.0.1:36416
I20260709 13:07:31.287866 17566 raft_consensus.cc:493] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.288239 17566 raft_consensus.cc:3060] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.294240 17566 raft_consensus.cc:515] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:31.296204 17566 leader_election.cc:290] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 3 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535)
I20260709 13:07:31.296963 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:31.297212 17492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "763080674d2b4bf384e4a5edfccd2ae8" candidate_uuid: "de549fcb09b7455eabcbf546a3604373" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
I20260709 13:07:31.297741 17765 raft_consensus.cc:3060] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.297792 17492 raft_consensus.cc:3055] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:31.298194 17492 raft_consensus.cc:740] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:31.299041 17492 consensus_queue.cc:260] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:31.300370 17492 raft_consensus.cc:3060] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.305385 17765 raft_consensus.cc:2468] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 3.
I20260709 13:07:31.306766 17531 leader_election.cc:304] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, de549fcb09b7455eabcbf546a3604373; no voters: 
I20260709 13:07:31.307550 17712 raft_consensus.cc:2804] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:31.307668 17492 raft_consensus.cc:2468] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de549fcb09b7455eabcbf546a3604373 in term 3.
I20260709 13:07:31.308168 17712 raft_consensus.cc:697] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Becoming Leader. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:31.309370 17712 consensus_queue.cc:237] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:31.317134 17313 catalog_manager.cc:5697] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 reported cstate change: term changed from 2 to 3, leader changed from e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66) to de549fcb09b7455eabcbf546a3604373 (127.7.213.67). New cstate: current_term: 3 leader_uuid: "de549fcb09b7455eabcbf546a3604373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.447558 17882 raft_consensus.cc:993] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373: : Instructing follower e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:31.448047 17839 raft_consensus.cc:1081] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Signalling peer e1ccf9084e5649ff824aae27144c3fa6 to start an election
I20260709 13:07:31.449527 17492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04"
dest_uuid: "e1ccf9084e5649ff824aae27144c3fa6"
 from {username='slave'} at 127.0.0.1:59808
I20260709 13:07:31.450060 17492 raft_consensus.cc:493] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.450340 17492 raft_consensus.cc:3060] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.454861 17492 raft_consensus.cc:515] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:31.456362 17492 leader_election.cc:290] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 3 election: Requested vote from peers 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125), de549fcb09b7455eabcbf546a3604373 (127.7.213.67:39751)
I20260709 13:07:31.457533 17566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "de549fcb09b7455eabcbf546a3604373"
I20260709 13:07:31.457557 17765 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fda897ab3f4d4e9fac83c26a24e4ac04" candidate_uuid: "e1ccf9084e5649ff824aae27144c3fa6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4ee5229f27334405a8010f6ee263c794"
I20260709 13:07:31.458241 17566 raft_consensus.cc:3055] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:31.458354 17765 raft_consensus.cc:3060] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.458628 17566 raft_consensus.cc:740] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: de549fcb09b7455eabcbf546a3604373, State: Running, Role: LEADER
I20260709 13:07:31.459638 17566 consensus_queue.cc:260] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:31.460950 17566 raft_consensus.cc:3060] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:31.464246 17765 raft_consensus.cc:2468] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 3.
I20260709 13:07:31.465543 17457 leader_election.cc:304] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ee5229f27334405a8010f6ee263c794, e1ccf9084e5649ff824aae27144c3fa6; no voters: 
I20260709 13:07:31.466250 17857 raft_consensus.cc:2804] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:31.466738 17857 raft_consensus.cc:697] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 LEADER]: Becoming Leader. State: Replica: e1ccf9084e5649ff824aae27144c3fa6, State: Running, Role: LEADER
I20260709 13:07:31.467453 17566 raft_consensus.cc:2468] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ccf9084e5649ff824aae27144c3fa6 in term 3.
I20260709 13:07:31.467816 17857 consensus_queue.cc:237] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } }
I20260709 13:07:31.474651 17313 catalog_manager.cc:5697] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 reported cstate change: term changed from 2 to 3, leader changed from de549fcb09b7455eabcbf546a3604373 (127.7.213.67) to e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "e1ccf9084e5649ff824aae27144c3fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:31.635586 17765 raft_consensus.cc:1275] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:31.637010 17883 consensus_queue.cc:1048] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:31.646838 17566 raft_consensus.cc:1275] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:31.650615 17883 consensus_queue.cc:1048] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:07:31.740402 17364 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:07:31.814795 17765 raft_consensus.cc:1275] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:31.816222 17882 consensus_queue.cc:1048] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:31.818696 17492 raft_consensus.cc:1275] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Refusing update from remote peer de549fcb09b7455eabcbf546a3604373: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:31.820232 17839 consensus_queue.cc:1048] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ccf9084e5649ff824aae27144c3fa6" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:31.861126 17566 raft_consensus.cc:1275] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:31.862280 17883 consensus_queue.cc:1048] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de549fcb09b7455eabcbf546a3604373" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 39751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:31.873759 17765 raft_consensus.cc:1275] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Refusing update from remote peer e1ccf9084e5649ff824aae27144c3fa6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:31.875332 17892 consensus_queue.cc:1048] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ee5229f27334405a8010f6ee263c794" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:32.004874  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:32.036162  8021 tablet_server.cc:179] TabletServer@127.7.213.65:42125 shutting down...
I20260709 13:07:32.059974  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:32.060694  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:32.061376  8021 raft_consensus.cc:2243] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.061841  8021 raft_consensus.cc:2272] T 34d273816f6244b78d341ddb9bec525e P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.065209  8021 tablet_replica.cc:333] stopping tablet replica
W20260709 13:07:32.065553 17457 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 74 similar messages]
I20260709 13:07:32.066092  8021 raft_consensus.cc:2243] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.066833  8021 raft_consensus.cc:2272] T fda897ab3f4d4e9fac83c26a24e4ac04 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:32.068729 17457 consensus_peers.cc:597] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.070086  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:32.070719  8021 raft_consensus.cc:2243] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.071105  8021 raft_consensus.cc:2272] T 5e7ecdf2c85e4be188b379c81de5d0b9 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.073370  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:32.073952  8021 raft_consensus.cc:2243] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.074497  8021 raft_consensus.cc:2272] T 8e57aa54c19f49c491ad03e6f4d42bf9 P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.076810  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:32.077399  8021 raft_consensus.cc:2243] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.077931  8021 raft_consensus.cc:2272] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.080193  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:32.080736  8021 raft_consensus.cc:2243] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.081187  8021 raft_consensus.cc:2272] T ab5343c89204428f9d6f571036f2f100 P 4ee5229f27334405a8010f6ee263c794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.083504  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:32.084161  8021 raft_consensus.cc:2243] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.084628  8021 raft_consensus.cc:2272] T 590942b106c341d1897bf550b0ff98a6 P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.086864  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:32.087370  8021 raft_consensus.cc:2243] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.087860  8021 raft_consensus.cc:2272] T 763080674d2b4bf384e4a5edfccd2ae8 P 4ee5229f27334405a8010f6ee263c794 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.090404  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:32.090875  8021 raft_consensus.cc:2243] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.091341  8021 raft_consensus.cc:2272] T 82ce26f2d71846ea8edacc70b2e04f6b P 4ee5229f27334405a8010f6ee263c794 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.093722  8021 tablet_replica.cc:333] stopping tablet replica
I20260709 13:07:32.094122  8021 raft_consensus.cc:2243] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.094530  8021 raft_consensus.cc:2272] T 25734691e67748a8955d13a8ef81f786 P 4ee5229f27334405a8010f6ee263c794 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.116379  8021 tablet_server.cc:196] TabletServer@127.7.213.65:42125 shutdown complete.
I20260709 13:07:32.144516  8021 tablet_server.cc:179] TabletServer@127.7.213.66:0 shutting down...
I20260709 13:07:32.176118  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:32.176888  8021 tablet_replica.cc:333] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.177722  8021 raft_consensus.cc:2243] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.178579  8021 raft_consensus.cc:2272] T 25734691e67748a8955d13a8ef81f786 P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.180826  8021 tablet_replica.cc:333] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.181494  8021 raft_consensus.cc:2243] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.182490  8021 raft_consensus.cc:2272] T 8e57aa54c19f49c491ad03e6f4d42bf9 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.184778  8021 tablet_replica.cc:333] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.185395  8021 raft_consensus.cc:2243] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.185839  8021 raft_consensus.cc:2272] T 5e7ecdf2c85e4be188b379c81de5d0b9 P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.187875  8021 tablet_replica.cc:333] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.188539  8021 raft_consensus.cc:2243] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.188999  8021 raft_consensus.cc:2272] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.191020  8021 tablet_replica.cc:333] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.191563  8021 raft_consensus.cc:2243] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.192108  8021 raft_consensus.cc:2272] T ab5343c89204428f9d6f571036f2f100 P e1ccf9084e5649ff824aae27144c3fa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.194484  8021 tablet_replica.cc:333] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.195096  8021 raft_consensus.cc:2243] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.195919  8021 raft_consensus.cc:2272] T fda897ab3f4d4e9fac83c26a24e4ac04 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.198963  8021 tablet_replica.cc:333] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.199532  8021 raft_consensus.cc:2243] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.200013  8021 raft_consensus.cc:2272] T 590942b106c341d1897bf550b0ff98a6 P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.202116  8021 tablet_replica.cc:333] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.202602  8021 raft_consensus.cc:2243] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.203011  8021 raft_consensus.cc:2272] T 763080674d2b4bf384e4a5edfccd2ae8 P e1ccf9084e5649ff824aae27144c3fa6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.205644  8021 tablet_replica.cc:333] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.206149  8021 raft_consensus.cc:2243] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.207015  8021 raft_consensus.cc:2272] T 82ce26f2d71846ea8edacc70b2e04f6b P e1ccf9084e5649ff824aae27144c3fa6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.209280  8021 tablet_replica.cc:333] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6: stopping tablet replica
I20260709 13:07:32.209758  8021 raft_consensus.cc:2243] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.210495  8021 raft_consensus.cc:2272] T 34d273816f6244b78d341ddb9bec525e P e1ccf9084e5649ff824aae27144c3fa6 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:32.225689 17533 consensus_peers.cc:597] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 -> Peer e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535): Couldn't send request to peer e1ccf9084e5649ff824aae27144c3fa6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.236694  8021 tablet_server.cc:196] TabletServer@127.7.213.66:0 shutdown complete.
W20260709 13:07:32.258460 17533 consensus_peers.cc:597] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 -> Peer e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535): Couldn't send request to peer e1ccf9084e5649ff824aae27144c3fa6. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:38535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:32.259014 17533 consensus_peers.cc:597] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 -> Peer e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535): Couldn't send request to peer e1ccf9084e5649ff824aae27144c3fa6. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:38535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.264599  8021 tablet_server.cc:179] TabletServer@127.7.213.67:0 shutting down...
W20260709 13:07:32.272171 17531 consensus_peers.cc:597] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 -> Peer 4ee5229f27334405a8010f6ee263c794 (127.7.213.65:42125): Couldn't send request to peer 4ee5229f27334405a8010f6ee263c794. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:42125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.296931  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:32.297714  8021 tablet_replica.cc:333] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.298367  8021 raft_consensus.cc:2243] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.299232  8021 raft_consensus.cc:2272] T 590942b106c341d1897bf550b0ff98a6 P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.302325  8021 tablet_replica.cc:333] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.303193  8021 raft_consensus.cc:2243] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.303731  8021 raft_consensus.cc:2272] T 34d273816f6244b78d341ddb9bec525e P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:32.307092 17533 consensus_peers.cc:597] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 -> Peer e1ccf9084e5649ff824aae27144c3fa6 (127.7.213.66:38535): Couldn't send request to peer e1ccf9084e5649ff824aae27144c3fa6. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:38535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.307687  8021 tablet_replica.cc:333] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.308324  8021 raft_consensus.cc:2243] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.309082  8021 raft_consensus.cc:2272] T 5e7ecdf2c85e4be188b379c81de5d0b9 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.311224  8021 tablet_replica.cc:333] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.311821  8021 raft_consensus.cc:2243] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.312278  8021 raft_consensus.cc:2272] T 8e57aa54c19f49c491ad03e6f4d42bf9 P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.314378  8021 tablet_replica.cc:333] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.314981  8021 raft_consensus.cc:2243] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.315846  8021 raft_consensus.cc:2272] T 5d63a3426265488cbfdaf0cc2f5cbbd7 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.318100  8021 tablet_replica.cc:333] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.318801  8021 raft_consensus.cc:2243] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.319675  8021 raft_consensus.cc:2272] T ab5343c89204428f9d6f571036f2f100 P de549fcb09b7455eabcbf546a3604373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.322038  8021 tablet_replica.cc:333] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.322595  8021 raft_consensus.cc:2243] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.323033  8021 raft_consensus.cc:2272] T 82ce26f2d71846ea8edacc70b2e04f6b P de549fcb09b7455eabcbf546a3604373 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.325317  8021 tablet_replica.cc:333] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.326025  8021 raft_consensus.cc:2243] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.327111  8021 raft_consensus.cc:2272] T 763080674d2b4bf384e4a5edfccd2ae8 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.331929  8021 tablet_replica.cc:333] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.332609  8021 raft_consensus.cc:2243] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.333099  8021 raft_consensus.cc:2272] T 25734691e67748a8955d13a8ef81f786 P de549fcb09b7455eabcbf546a3604373 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.335155  8021 tablet_replica.cc:333] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373: stopping tablet replica
I20260709 13:07:32.335604  8021 raft_consensus.cc:2243] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.336066  8021 raft_consensus.cc:2272] T fda897ab3f4d4e9fac83c26a24e4ac04 P de549fcb09b7455eabcbf546a3604373 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.367870  8021 tablet_server.cc:196] TabletServer@127.7.213.67:0 shutdown complete.
I20260709 13:07:32.393843  8021 master.cc:562] Master@127.7.213.126:35643 shutting down...
I20260709 13:07:32.428100  8021 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.428653  8021 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.428952  8021 tablet_replica.cc:333] T 00000000000000000000000000000000 P 53474a5fdc5444ccb775be32b9b339c8: stopping tablet replica
I20260709 13:07:32.450806  8021 master.cc:584] Master@127.7.213.126:35643 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12902 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:07:32.498135  8021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.213.126:33675
I20260709 13:07:32.499361  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:32.505497 17899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:32.506074 17898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:32.506410 17901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:32.506983  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:32.507911  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:32.508082  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:32.508198  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602452508187 us; error 0 us; skew 500 ppm
I20260709 13:07:32.508666  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:32.510918  8021 webserver.cc:533] Webserver started at http://127.7.213.126:36967/ using document root <none> and password file <none>
I20260709 13:07:32.511370  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:32.511523  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:32.511797  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:32.513007  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/master-0-root/instance:
uuid: "6f6b715ed3524ea08c8d04c9ec470892"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-9g29"
I20260709 13:07:32.517473  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:32.520872 17906 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.521773  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:07:32.522104  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/master-0-root
uuid: "6f6b715ed3524ea08c8d04c9ec470892"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-9g29"
I20260709 13:07:32.522403  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:32.535861  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:32.537087  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:32.575552  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.126:33675
I20260709 13:07:32.575877 17957 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.126:33675 every 8 connection(s)
I20260709 13:07:32.579623 17958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.591354 17958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892: Bootstrap starting.
I20260709 13:07:32.596252 17958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.600416 17958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892: No bootstrap required, opened a new log
I20260709 13:07:32.602541 17958 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6b715ed3524ea08c8d04c9ec470892" member_type: VOTER }
I20260709 13:07:32.602946 17958 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.603204 17958 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f6b715ed3524ea08c8d04c9ec470892, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.603790 17958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6b715ed3524ea08c8d04c9ec470892" member_type: VOTER }
I20260709 13:07:32.604262 17958 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:32.604514 17958 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:32.604776 17958 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.609498 17958 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6b715ed3524ea08c8d04c9ec470892" member_type: VOTER }
I20260709 13:07:32.610031 17958 leader_election.cc:304] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f6b715ed3524ea08c8d04c9ec470892; no voters: 
I20260709 13:07:32.611137 17958 leader_election.cc:290] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:32.611552 17961 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.612924 17961 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 1 LEADER]: Becoming Leader. State: Replica: 6f6b715ed3524ea08c8d04c9ec470892, State: Running, Role: LEADER
I20260709 13:07:32.613505 17961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6b715ed3524ea08c8d04c9ec470892" member_type: VOTER }
I20260709 13:07:32.614107 17958 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:32.616416 17962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f6b715ed3524ea08c8d04c9ec470892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6b715ed3524ea08c8d04c9ec470892" member_type: VOTER } }
I20260709 13:07:32.616618 17963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f6b715ed3524ea08c8d04c9ec470892. Latest consensus state: current_term: 1 leader_uuid: "6f6b715ed3524ea08c8d04c9ec470892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f6b715ed3524ea08c8d04c9ec470892" member_type: VOTER } }
I20260709 13:07:32.617113 17962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:32.617244 17963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:32.619782 17967 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:32.628006 17967 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:32.629528  8021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:32.636957 17967 catalog_manager.cc:1383] Generated new cluster ID: 23fc9f01249c48bd8b2aee6a9f711ba9
I20260709 13:07:32.637261 17967 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:32.648262 17967 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:32.649668 17967 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:32.667358 17967 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892: Generated new TSK 0
I20260709 13:07:32.668043 17967 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:32.696175  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:32.702543 17979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:32.703372 17980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:32.706075  8021 server_base.cc:1061] running on GCE node
W20260709 13:07:32.705982 17982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:32.707512  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:32.707805  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:32.708002  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602452707987 us; error 0 us; skew 500 ppm
I20260709 13:07:32.708534  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:32.710920  8021 webserver.cc:533] Webserver started at http://127.7.213.65:45411/ using document root <none> and password file <none>
I20260709 13:07:32.711404  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:32.711611  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:32.711907  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:32.713027  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-0-root/instance:
uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-9g29"
I20260709 13:07:32.717701  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:32.720956 17987 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.721917  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:32.722216  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-9g29"
I20260709 13:07:32.722482  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:32.738263  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:32.739550  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:32.741142  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:32.743714  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:32.744032  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.744288  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:32.744501  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.787065  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:46449
I20260709 13:07:32.787171 18049 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:46449 every 8 connection(s)
I20260709 13:07:32.792313  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:32.799654 18054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:32.801194 18055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:32.804181  8021 server_base.cc:1061] running on GCE node
W20260709 13:07:32.804205 18057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:32.805619  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:32.805888  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:32.806089  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602452806070 us; error 0 us; skew 500 ppm
I20260709 13:07:32.806627  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:32.807104 18050 heartbeater.cc:344] Connected to a master server at 127.7.213.126:33675
I20260709 13:07:32.807616 18050 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:32.808530 18050 heartbeater.cc:507] Master 127.7.213.126:33675 requested a full tablet report, sending...
I20260709 13:07:32.809374  8021 webserver.cc:533] Webserver started at http://127.7.213.66:46865/ using document root <none> and password file <none>
I20260709 13:07:32.810004  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:32.810217  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:32.810487  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:32.811085 17923 ts_manager.cc:194] Registered new tserver with Master: 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65:46449)
I20260709 13:07:32.811789  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-1-root/instance:
uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-9g29"
I20260709 13:07:32.813031 17923 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54188
I20260709 13:07:32.816936  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:32.820503 18062 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.821393  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:32.821705  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-1-root
uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-9g29"
I20260709 13:07:32.821971  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:32.839655  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:32.840925  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:32.842378  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:32.844619  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:32.844914  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.001s
I20260709 13:07:32.845122  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:32.845257  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.884483  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.66:43325
I20260709 13:07:32.884650 18124 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.66:43325 every 8 connection(s)
I20260709 13:07:32.889308  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:32.896399 18128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:32.899163 18129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:32.901049  8021 server_base.cc:1061] running on GCE node
W20260709 13:07:32.901165 18131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:32.902618 18125 heartbeater.cc:344] Connected to a master server at 127.7.213.126:33675
I20260709 13:07:32.902717  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:32.903067  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:32.903069 18125 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:32.903350  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602452903333 us; error 0 us; skew 500 ppm
I20260709 13:07:32.904102 18125 heartbeater.cc:507] Master 127.7.213.126:33675 requested a full tablet report, sending...
I20260709 13:07:32.904114  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:32.906459 17923 ts_manager.cc:194] Registered new tserver with Master: 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66:43325)
I20260709 13:07:32.907197  8021 webserver.cc:533] Webserver started at http://127.7.213.67:46549/ using document root <none> and password file <none>
I20260709 13:07:32.907918  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:32.908167  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:32.908473  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:32.908424 17923 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54202
I20260709 13:07:32.910143  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-2-root/instance:
uuid: "62de25923ada4882a388f560a3934a0b"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-9g29"
I20260709 13:07:32.915239  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:07:32.918756 18136 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.919544  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:32.919921  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-2-root
uuid: "62de25923ada4882a388f560a3934a0b"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-9g29"
I20260709 13:07:32.920220  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602158212224-8021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:32.930119  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:32.931383  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:32.932921  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:32.935253  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:32.935480  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.935778  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:32.935969  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:07:32.979178  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.67:41111
I20260709 13:07:32.979292 18198 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.67:41111 every 8 connection(s)
I20260709 13:07:32.994031 18199 heartbeater.cc:344] Connected to a master server at 127.7.213.126:33675
I20260709 13:07:32.994519 18199 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:32.995390 18199 heartbeater.cc:507] Master 127.7.213.126:33675 requested a full tablet report, sending...
I20260709 13:07:32.997484 17923 ts_manager.cc:194] Registered new tserver with Master: 62de25923ada4882a388f560a3934a0b (127.7.213.67:41111)
I20260709 13:07:32.998589  8021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015967306s
I20260709 13:07:32.998883 17923 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54214
I20260709 13:07:33.028569 17923 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54230:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:33.030953 17923 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:33.072484 18090 tablet_service.cc:1511] Processing CreateTablet for tablet 157ba30eb6b74f38a760e54abcda90f9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=007cfcc1767943f5b4f67468cde81f35]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:33.072829 18089 tablet_service.cc:1511] Processing CreateTablet for tablet e3ea749ddf6a48d8b31aefb041f177e0 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=007cfcc1767943f5b4f67468cde81f35]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:33.074054 18090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 157ba30eb6b74f38a760e54abcda90f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.074739 18089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3ea749ddf6a48d8b31aefb041f177e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.083524 18015 tablet_service.cc:1511] Processing CreateTablet for tablet 157ba30eb6b74f38a760e54abcda90f9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=007cfcc1767943f5b4f67468cde81f35]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:33.085057 18015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 157ba30eb6b74f38a760e54abcda90f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.088191 18164 tablet_service.cc:1511] Processing CreateTablet for tablet 157ba30eb6b74f38a760e54abcda90f9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=007cfcc1767943f5b4f67468cde81f35]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:33.089634 18164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 157ba30eb6b74f38a760e54abcda90f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.092170 18163 tablet_service.cc:1511] Processing CreateTablet for tablet e3ea749ddf6a48d8b31aefb041f177e0 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=007cfcc1767943f5b4f67468cde81f35]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:33.093624 18163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3ea749ddf6a48d8b31aefb041f177e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.093005 18014 tablet_service.cc:1511] Processing CreateTablet for tablet e3ea749ddf6a48d8b31aefb041f177e0 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=007cfcc1767943f5b4f67468cde81f35]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:33.094909 18014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3ea749ddf6a48d8b31aefb041f177e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.101854 18214 tablet_bootstrap.cc:492] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Bootstrap starting.
I20260709 13:07:33.114620 18214 tablet_bootstrap.cc:654] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.125916 18215 tablet_bootstrap.cc:492] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b: Bootstrap starting.
I20260709 13:07:33.128243 18217 tablet_bootstrap.cc:492] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b: Bootstrap starting.
I20260709 13:07:33.135390 18217 tablet_bootstrap.cc:654] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.136232 18215 tablet_bootstrap.cc:654] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.137920 18214 tablet_bootstrap.cc:492] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3: No bootstrap required, opened a new log
I20260709 13:07:33.138617 18214 ts_tablet_manager.cc:1403] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Time spent bootstrapping tablet: real 0.037s	user 0.025s	sys 0.004s
I20260709 13:07:33.141896 18214 raft_consensus.cc:359] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.142552 18217 tablet_bootstrap.cc:492] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b: No bootstrap required, opened a new log
I20260709 13:07:33.142652 18214 raft_consensus.cc:385] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.143069 18214 raft_consensus.cc:740] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45afcea9a9b34c609e3f38bcc7f9ebe3, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.143127 18217 ts_tablet_manager.cc:1403] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b: Time spent bootstrapping tablet: real 0.015s	user 0.002s	sys 0.009s
I20260709 13:07:33.143265 18215 tablet_bootstrap.cc:492] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b: No bootstrap required, opened a new log
I20260709 13:07:33.143914 18215 ts_tablet_manager.cc:1403] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.008s
I20260709 13:07:33.143975 18214 consensus_queue.cc:260] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.146608 18125 heartbeater.cc:499] Master 127.7.213.126:33675 was elected leader, sending a full tablet report...
I20260709 13:07:33.147203 18214 ts_tablet_manager.cc:1434] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:33.146709 18215 raft_consensus.cc:359] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } }
I20260709 13:07:33.148324 18214 tablet_bootstrap.cc:492] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Bootstrap starting.
I20260709 13:07:33.148290 18215 raft_consensus.cc:385] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.148814 18215 raft_consensus.cc:740] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62de25923ada4882a388f560a3934a0b, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.146550 18217 raft_consensus.cc:359] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } }
I20260709 13:07:33.149597 18217 raft_consensus.cc:385] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.149907 18217 raft_consensus.cc:740] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e89b014bea41e5b4a48a4bdc95f53b, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.150749 18217 consensus_queue.cc:260] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } }
I20260709 13:07:33.152084 18215 consensus_queue.cc:260] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } }
I20260709 13:07:33.157141 18214 tablet_bootstrap.cc:654] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.161918 18215 ts_tablet_manager.cc:1434] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b: Time spent starting tablet: real 0.018s	user 0.012s	sys 0.001s
I20260709 13:07:33.162801 18199 heartbeater.cc:499] Master 127.7.213.126:33675 was elected leader, sending a full tablet report...
I20260709 13:07:33.166530 18050 heartbeater.cc:499] Master 127.7.213.126:33675 was elected leader, sending a full tablet report...
I20260709 13:07:33.167557 18217 ts_tablet_manager.cc:1434] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b: Time spent starting tablet: real 0.024s	user 0.013s	sys 0.010s
I20260709 13:07:33.168977 18217 tablet_bootstrap.cc:492] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b: Bootstrap starting.
I20260709 13:07:33.169873 18215 tablet_bootstrap.cc:492] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b: Bootstrap starting.
I20260709 13:07:33.180334 18217 tablet_bootstrap.cc:654] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.181484 18215 tablet_bootstrap.cc:654] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.187316 18214 tablet_bootstrap.cc:492] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3: No bootstrap required, opened a new log
I20260709 13:07:33.187330 18217 tablet_bootstrap.cc:492] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b: No bootstrap required, opened a new log
I20260709 13:07:33.187618 18215 tablet_bootstrap.cc:492] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b: No bootstrap required, opened a new log
I20260709 13:07:33.188009 18214 ts_tablet_manager.cc:1403] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Time spent bootstrapping tablet: real 0.040s	user 0.035s	sys 0.004s
I20260709 13:07:33.188061 18217 ts_tablet_manager.cc:1403] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.001s
I20260709 13:07:33.188149 18215 ts_tablet_manager.cc:1403] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.005s
I20260709 13:07:33.190837 18217 raft_consensus.cc:359] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.191392 18217 raft_consensus.cc:385] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.191293 18214 raft_consensus.cc:359] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } }
I20260709 13:07:33.191846 18217 raft_consensus.cc:740] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e89b014bea41e5b4a48a4bdc95f53b, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.191706 18215 raft_consensus.cc:359] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.192189 18214 raft_consensus.cc:385] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.192597 18215 raft_consensus.cc:385] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.192647 18214 raft_consensus.cc:740] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45afcea9a9b34c609e3f38bcc7f9ebe3, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.192910 18215 raft_consensus.cc:740] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62de25923ada4882a388f560a3934a0b, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.192898 18217 consensus_queue.cc:260] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.193553 18214 consensus_queue.cc:260] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } }
I20260709 13:07:33.193909 18215 consensus_queue.cc:260] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.195294 18217 ts_tablet_manager.cc:1434] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:07:33.195677 18214 ts_tablet_manager.cc:1434] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:07:33.197856 18215 ts_tablet_manager.cc:1434] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:07:33.300122 18220 raft_consensus.cc:493] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.300700 18220 raft_consensus.cc:515] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.302834 18220 leader_election.cc:290] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65:46449), 62de25923ada4882a388f560a3934a0b (127.7.213.67:41111)
I20260709 13:07:33.313053 18025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ea749ddf6a48d8b31aefb041f177e0" candidate_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" is_pre_election: true
I20260709 13:07:33.313764 18025 raft_consensus.cc:2468] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45afcea9a9b34c609e3f38bcc7f9ebe3 in term 0.
I20260709 13:07:33.314143 18174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ea749ddf6a48d8b31aefb041f177e0" candidate_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62de25923ada4882a388f560a3934a0b" is_pre_election: true
I20260709 13:07:33.314867 18065 leader_election.cc:304] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45afcea9a9b34c609e3f38bcc7f9ebe3, 49e89b014bea41e5b4a48a4bdc95f53b; no voters: 
I20260709 13:07:33.314862 18174 raft_consensus.cc:2468] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45afcea9a9b34c609e3f38bcc7f9ebe3 in term 0.
I20260709 13:07:33.315907 18220 raft_consensus.cc:2804] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:33.316228 18220 raft_consensus.cc:493] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.316512 18220 raft_consensus.cc:3060] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.322047 18220 raft_consensus.cc:515] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.323794 18220 leader_election.cc:290] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [CANDIDATE]: Term 1 election: Requested vote from peers 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65:46449), 62de25923ada4882a388f560a3934a0b (127.7.213.67:41111)
I20260709 13:07:33.324594 18025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ea749ddf6a48d8b31aefb041f177e0" candidate_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
I20260709 13:07:33.325054 18174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3ea749ddf6a48d8b31aefb041f177e0" candidate_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62de25923ada4882a388f560a3934a0b"
I20260709 13:07:33.325315 18025 raft_consensus.cc:3060] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.325671 18174 raft_consensus.cc:3060] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.330730 18025 raft_consensus.cc:2468] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45afcea9a9b34c609e3f38bcc7f9ebe3 in term 1.
I20260709 13:07:33.331103 18174 raft_consensus.cc:2468] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45afcea9a9b34c609e3f38bcc7f9ebe3 in term 1.
I20260709 13:07:33.331864 18065 leader_election.cc:304] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45afcea9a9b34c609e3f38bcc7f9ebe3, 49e89b014bea41e5b4a48a4bdc95f53b; no voters: 
I20260709 13:07:33.332592 18220 raft_consensus.cc:2804] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:33.333504 18220 raft_consensus.cc:697] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 LEADER]: Becoming Leader. State: Replica: 45afcea9a9b34c609e3f38bcc7f9ebe3, State: Running, Role: LEADER
I20260709 13:07:33.334280 18220 consensus_queue.cc:237] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.340488 17922 catalog_manager.cc:5697] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:33.431655 18221 raft_consensus.cc:493] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.432325 18221 raft_consensus.cc:515] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } }
I20260709 13:07:33.434448 18221 leader_election.cc:290] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66:43325), 62de25923ada4882a388f560a3934a0b (127.7.213.67:41111)
I20260709 13:07:33.445235 18100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157ba30eb6b74f38a760e54abcda90f9" candidate_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" is_pre_election: true
I20260709 13:07:33.445350 18174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157ba30eb6b74f38a760e54abcda90f9" candidate_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62de25923ada4882a388f560a3934a0b" is_pre_election: true
I20260709 13:07:33.445955 18100 raft_consensus.cc:2468] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49e89b014bea41e5b4a48a4bdc95f53b in term 0.
I20260709 13:07:33.445972 18174 raft_consensus.cc:2468] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49e89b014bea41e5b4a48a4bdc95f53b in term 0.
I20260709 13:07:33.447080 17990 leader_election.cc:304] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45afcea9a9b34c609e3f38bcc7f9ebe3, 49e89b014bea41e5b4a48a4bdc95f53b; no voters: 
I20260709 13:07:33.447813 18221 raft_consensus.cc:2804] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:33.448191 18221 raft_consensus.cc:493] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.448413 18221 raft_consensus.cc:3060] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.453408 18221 raft_consensus.cc:515] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } }
I20260709 13:07:33.454918 18221 leader_election.cc:290] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [CANDIDATE]: Term 1 election: Requested vote from peers 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66:43325), 62de25923ada4882a388f560a3934a0b (127.7.213.67:41111)
I20260709 13:07:33.455797 18100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157ba30eb6b74f38a760e54abcda90f9" candidate_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3"
I20260709 13:07:33.455948 18174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "157ba30eb6b74f38a760e54abcda90f9" candidate_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62de25923ada4882a388f560a3934a0b"
I20260709 13:07:33.456615 18100 raft_consensus.cc:3060] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.456679 18174 raft_consensus.cc:3060] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.464066 18174 raft_consensus.cc:2468] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e89b014bea41e5b4a48a4bdc95f53b in term 1.
I20260709 13:07:33.464066 18100 raft_consensus.cc:2468] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e89b014bea41e5b4a48a4bdc95f53b in term 1.
I20260709 13:07:33.465482 17988 leader_election.cc:304] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e89b014bea41e5b4a48a4bdc95f53b, 62de25923ada4882a388f560a3934a0b; no voters: 
I20260709 13:07:33.466167 18221 raft_consensus.cc:2804] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:33.467352 18221 raft_consensus.cc:697] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 LEADER]: Becoming Leader. State: Replica: 49e89b014bea41e5b4a48a4bdc95f53b, State: Running, Role: LEADER
I20260709 13:07:33.468170 18221 consensus_queue.cc:237] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } }
I20260709 13:07:33.474963 17922 catalog_manager.cc:5697] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b reported cstate change: term changed from 0 to 1, leader changed from <none> to 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65). New cstate: current_term: 1 leader_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:33.570879 17922 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54244:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:33.573365 17922 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:33.599097 18014 tablet_service.cc:1511] Processing CreateTablet for tablet 78da03bd48154a5197984894e4dddf69 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=199a14507d2341988f4490351b15368a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:33.599889 18089 tablet_service.cc:1511] Processing CreateTablet for tablet 78da03bd48154a5197984894e4dddf69 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=199a14507d2341988f4490351b15368a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:33.600628 18014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78da03bd48154a5197984894e4dddf69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.601361 18089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78da03bd48154a5197984894e4dddf69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.603153 18015 tablet_service.cc:1511] Processing CreateTablet for tablet 94936d8fcec34871beb274ec4e451f5d (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=199a14507d2341988f4490351b15368a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:33.604543 18015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94936d8fcec34871beb274ec4e451f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.604764 18090 tablet_service.cc:1511] Processing CreateTablet for tablet 94936d8fcec34871beb274ec4e451f5d (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=199a14507d2341988f4490351b15368a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:33.605672 18164 tablet_service.cc:1511] Processing CreateTablet for tablet 78da03bd48154a5197984894e4dddf69 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=199a14507d2341988f4490351b15368a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:07:33.606096 18090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94936d8fcec34871beb274ec4e451f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.607028 18164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78da03bd48154a5197984894e4dddf69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.618089 18163 tablet_service.cc:1511] Processing CreateTablet for tablet 94936d8fcec34871beb274ec4e451f5d (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=199a14507d2341988f4490351b15368a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:07:33.619632 18163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94936d8fcec34871beb274ec4e451f5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.622759 18217 tablet_bootstrap.cc:492] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b: Bootstrap starting.
W20260709 13:07:33.624688 17971 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:33.631248 18217 tablet_bootstrap.cc:654] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.633675 18214 tablet_bootstrap.cc:492] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3: Bootstrap starting.
I20260709 13:07:33.637547 18215 tablet_bootstrap.cc:492] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b: Bootstrap starting.
I20260709 13:07:33.643859 18214 tablet_bootstrap.cc:654] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.645340 18215 tablet_bootstrap.cc:654] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.648248 18217 tablet_bootstrap.cc:492] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b: No bootstrap required, opened a new log
I20260709 13:07:33.649350 18217 ts_tablet_manager.cc:1403] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.008s
I20260709 13:07:33.654805 18217 raft_consensus.cc:359] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.658893 18217 raft_consensus.cc:385] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.658187 18214 tablet_bootstrap.cc:492] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3: No bootstrap required, opened a new log
I20260709 13:07:33.659292 18217 raft_consensus.cc:740] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e89b014bea41e5b4a48a4bdc95f53b, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.659791 18214 ts_tablet_manager.cc:1403] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.008s
I20260709 13:07:33.660182 18217 consensus_queue.cc:260] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.661226 18215 tablet_bootstrap.cc:492] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b: No bootstrap required, opened a new log
I20260709 13:07:33.663095 18215 ts_tablet_manager.cc:1403] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.008s
I20260709 13:07:33.663113 18217 ts_tablet_manager.cc:1434] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:07:33.663196 18214 raft_consensus.cc:359] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.664248 18214 raft_consensus.cc:385] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.664654 18214 raft_consensus.cc:740] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45afcea9a9b34c609e3f38bcc7f9ebe3, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.664932 18217 tablet_bootstrap.cc:492] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b: Bootstrap starting.
I20260709 13:07:33.665676 18214 consensus_queue.cc:260] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.668627 18214 ts_tablet_manager.cc:1434] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:07:33.666697 18215 raft_consensus.cc:359] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.669673 18215 raft_consensus.cc:385] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.670040 18215 raft_consensus.cc:740] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62de25923ada4882a388f560a3934a0b, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.670120 18214 tablet_bootstrap.cc:492] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Bootstrap starting.
I20260709 13:07:33.671168 18215 consensus_queue.cc:260] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.673110 18217 tablet_bootstrap.cc:654] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.674394 18222 raft_consensus.cc:493] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.674958 18222 raft_consensus.cc:515] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.677932 18222 leader_election.cc:290] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65:46449), 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66:43325)
I20260709 13:07:33.684019 18215 ts_tablet_manager.cc:1434] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:07:33.685331 18215 tablet_bootstrap.cc:492] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b: Bootstrap starting.
I20260709 13:07:33.694473 18215 tablet_bootstrap.cc:654] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.702374 18214 tablet_bootstrap.cc:654] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.702786 18217 tablet_bootstrap.cc:492] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b: No bootstrap required, opened a new log
I20260709 13:07:33.703395 18217 ts_tablet_manager.cc:1403] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b: Time spent bootstrapping tablet: real 0.039s	user 0.021s	sys 0.012s
I20260709 13:07:33.706907 18217 raft_consensus.cc:359] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.708016 18217 raft_consensus.cc:385] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.708511 18217 raft_consensus.cc:740] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e89b014bea41e5b4a48a4bdc95f53b, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.709762 18217 consensus_queue.cc:260] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.712154 18215 tablet_bootstrap.cc:492] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b: No bootstrap required, opened a new log
I20260709 13:07:33.712828 18215 ts_tablet_manager.cc:1403] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.000s
I20260709 13:07:33.712960 18217 ts_tablet_manager.cc:1434] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:07:33.716357 18214 tablet_bootstrap.cc:492] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3: No bootstrap required, opened a new log
I20260709 13:07:33.716964 18214 ts_tablet_manager.cc:1403] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.007s
I20260709 13:07:33.716596 18215 raft_consensus.cc:359] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.717425 18215 raft_consensus.cc:385] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.717780 18025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78da03bd48154a5197984894e4dddf69" candidate_uuid: "62de25923ada4882a388f560a3934a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" is_pre_election: true
I20260709 13:07:33.717828 18215 raft_consensus.cc:740] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62de25923ada4882a388f560a3934a0b, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.718681 18025 raft_consensus.cc:2468] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62de25923ada4882a388f560a3934a0b in term 0.
I20260709 13:07:33.719312 18215 consensus_queue.cc:260] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.720480 18214 raft_consensus.cc:359] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.721319 18214 raft_consensus.cc:385] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.721647 18214 raft_consensus.cc:740] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45afcea9a9b34c609e3f38bcc7f9ebe3, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.722344 18215 ts_tablet_manager.cc:1434] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:07:33.722651 18214 consensus_queue.cc:260] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.726140 18139 leader_election.cc:304] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e89b014bea41e5b4a48a4bdc95f53b, 62de25923ada4882a388f560a3934a0b; no voters: 
I20260709 13:07:33.729326 18214 ts_tablet_manager.cc:1434] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3: Time spent starting tablet: real 0.012s	user 0.012s	sys 0.000s
I20260709 13:07:33.729149 18100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78da03bd48154a5197984894e4dddf69" candidate_uuid: "62de25923ada4882a388f560a3934a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" is_pre_election: true
I20260709 13:07:33.730013 18100 raft_consensus.cc:2468] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62de25923ada4882a388f560a3934a0b in term 0.
I20260709 13:07:33.730389 18222 raft_consensus.cc:2804] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:33.730913 18222 raft_consensus.cc:493] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:33.731307 18222 raft_consensus.cc:3060] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.739925 18220 consensus_queue.cc:1048] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:33.743825 18222 raft_consensus.cc:515] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.747153 18025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78da03bd48154a5197984894e4dddf69" candidate_uuid: "62de25923ada4882a388f560a3934a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
I20260709 13:07:33.747816 18025 raft_consensus.cc:3060] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.748692 18222 leader_election.cc:290] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [CANDIDATE]: Term 1 election: Requested vote from peers 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65:46449), 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66:43325)
I20260709 13:07:33.747685 18100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78da03bd48154a5197984894e4dddf69" candidate_uuid: "62de25923ada4882a388f560a3934a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3"
I20260709 13:07:33.749544 18100 raft_consensus.cc:3060] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:33.755681 18025 raft_consensus.cc:2468] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62de25923ada4882a388f560a3934a0b in term 1.
I20260709 13:07:33.757109 18139 leader_election.cc:304] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e89b014bea41e5b4a48a4bdc95f53b, 62de25923ada4882a388f560a3934a0b; no voters: 
I20260709 13:07:33.758561 18247 raft_consensus.cc:2804] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:33.759212 18247 raft_consensus.cc:697] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 1 LEADER]: Becoming Leader. State: Replica: 62de25923ada4882a388f560a3934a0b, State: Running, Role: LEADER
I20260709 13:07:33.760213 18247 consensus_queue.cc:237] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:33.763497 18220 consensus_queue.cc:1048] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:33.774325 18100 raft_consensus.cc:2468] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62de25923ada4882a388f560a3934a0b in term 1.
I20260709 13:07:33.783150 17921 catalog_manager.cc:5697] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 62de25923ada4882a388f560a3934a0b (127.7.213.67). New cstate: current_term: 1 leader_uuid: "62de25923ada4882a388f560a3934a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:34.011117 18221 consensus_queue.cc:1048] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:34.019557 18257 consensus_queue.cc:1048] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:34.029603 18220 raft_consensus.cc:493] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.030134 18220 raft_consensus.cc:515] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:34.032327 18220 leader_election.cc:290] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65:46449), 62de25923ada4882a388f560a3934a0b (127.7.213.67:41111)
I20260709 13:07:34.033036 18025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d" candidate_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" is_pre_election: true
I20260709 13:07:34.033541 18174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d" candidate_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62de25923ada4882a388f560a3934a0b" is_pre_election: true
I20260709 13:07:34.033772 18025 raft_consensus.cc:2468] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45afcea9a9b34c609e3f38bcc7f9ebe3 in term 0.
I20260709 13:07:34.034205 18174 raft_consensus.cc:2468] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45afcea9a9b34c609e3f38bcc7f9ebe3 in term 0.
I20260709 13:07:34.035070 18065 leader_election.cc:304] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45afcea9a9b34c609e3f38bcc7f9ebe3, 49e89b014bea41e5b4a48a4bdc95f53b; no voters: 
I20260709 13:07:34.035801 18220 raft_consensus.cc:2804] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:34.036175 18220 raft_consensus.cc:493] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.036453 18220 raft_consensus.cc:3060] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:34.041906 18220 raft_consensus.cc:515] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:34.043503 18220 leader_election.cc:290] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [CANDIDATE]: Term 1 election: Requested vote from peers 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65:46449), 62de25923ada4882a388f560a3934a0b (127.7.213.67:41111)
I20260709 13:07:34.044183 18025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d" candidate_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
I20260709 13:07:34.044569 18174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d" candidate_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62de25923ada4882a388f560a3934a0b"
I20260709 13:07:34.044893 18025 raft_consensus.cc:3060] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:34.045177 18174 raft_consensus.cc:3060] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:34.052040 18174 raft_consensus.cc:2468] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45afcea9a9b34c609e3f38bcc7f9ebe3 in term 1.
I20260709 13:07:34.052040 18025 raft_consensus.cc:2468] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45afcea9a9b34c609e3f38bcc7f9ebe3 in term 1.
I20260709 13:07:34.053406 18063 leader_election.cc:304] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45afcea9a9b34c609e3f38bcc7f9ebe3, 62de25923ada4882a388f560a3934a0b; no voters: 
I20260709 13:07:34.054111 18220 raft_consensus.cc:2804] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:34.054554 18220 raft_consensus.cc:697] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 LEADER]: Becoming Leader. State: Replica: 45afcea9a9b34c609e3f38bcc7f9ebe3, State: Running, Role: LEADER
I20260709 13:07:34.055203 18220 consensus_queue.cc:237] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:34.062752 17920 catalog_manager.cc:5697] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.141319 18174 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78da03bd48154a5197984894e4dddf69"
dest_uuid: "62de25923ada4882a388f560a3934a0b"
mode: GRACEFUL
new_leader_uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:07:34.141826 18174 raft_consensus.cc:606] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 1 LEADER]: Received request to transfer leadership to TS 49e89b014bea41e5b4a48a4bdc95f53b
I20260709 13:07:34.154222 18100 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d"
dest_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3"
mode: GRACEFUL
new_leader_uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
 from {username='slave'} at 127.0.0.1:48048
I20260709 13:07:34.154759 18100 raft_consensus.cc:606] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 LEADER]: Received request to transfer leadership to TS 49e89b014bea41e5b4a48a4bdc95f53b
I20260709 13:07:34.167980 18222 consensus_queue.cc:1048] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:34.175118 18247 raft_consensus.cc:993] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b: : Instructing follower 49e89b014bea41e5b4a48a4bdc95f53b to start an election
I20260709 13:07:34.175957 18222 raft_consensus.cc:1081] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 1 LEADER]: Signalling peer 49e89b014bea41e5b4a48a4bdc95f53b to start an election
I20260709 13:07:34.178861 18024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78da03bd48154a5197984894e4dddf69"
dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
 from {username='slave'} at 127.0.0.1:50268
I20260709 13:07:34.180460 18024 raft_consensus.cc:493] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:34.180856 18024 raft_consensus.cc:3060] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:34.182210 18247 consensus_queue.cc:1048] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:34.188882 18024 raft_consensus.cc:515] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:34.194137 18024 leader_election.cc:290] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [CANDIDATE]: Term 2 election: Requested vote from peers 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66:43325), 62de25923ada4882a388f560a3934a0b (127.7.213.67:41111)
I20260709 13:07:34.195174 18099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78da03bd48154a5197984894e4dddf69" candidate_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3"
I20260709 13:07:34.195837 18099 raft_consensus.cc:3060] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:34.196930 18174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78da03bd48154a5197984894e4dddf69" candidate_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62de25923ada4882a388f560a3934a0b"
I20260709 13:07:34.197548 18174 raft_consensus.cc:3055] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:34.197822 18174 raft_consensus.cc:740] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 62de25923ada4882a388f560a3934a0b, State: Running, Role: LEADER
I20260709 13:07:34.198525 18174 consensus_queue.cc:260] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:34.199298 18174 raft_consensus.cc:3060] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:34.202176 18099 raft_consensus.cc:2468] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e89b014bea41e5b4a48a4bdc95f53b in term 2.
I20260709 13:07:34.203184 17990 leader_election.cc:304] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45afcea9a9b34c609e3f38bcc7f9ebe3, 49e89b014bea41e5b4a48a4bdc95f53b; no voters: 
I20260709 13:07:34.204011 18257 raft_consensus.cc:2804] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:34.204494 18257 raft_consensus.cc:697] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 LEADER]: Becoming Leader. State: Replica: 49e89b014bea41e5b4a48a4bdc95f53b, State: Running, Role: LEADER
I20260709 13:07:34.205130 18174 raft_consensus.cc:2468] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e89b014bea41e5b4a48a4bdc95f53b in term 2.
I20260709 13:07:34.205166 18257 consensus_queue.cc:237] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:34.212162 17919 catalog_manager.cc:5697] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b reported cstate change: term changed from 1 to 2, leader changed from 62de25923ada4882a388f560a3934a0b (127.7.213.67) to 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65). New cstate: current_term: 2 leader_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.555697 18220 consensus_queue.cc:1048] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:34.568809 18266 consensus_queue.cc:1048] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:34.577443 18251 raft_consensus.cc:993] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3: : Instructing follower 49e89b014bea41e5b4a48a4bdc95f53b to start an election
I20260709 13:07:34.577961 18266 raft_consensus.cc:1081] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 LEADER]: Signalling peer 49e89b014bea41e5b4a48a4bdc95f53b to start an election
I20260709 13:07:34.579873 18024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d"
dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
 from {username='slave'} at 127.0.0.1:50254
I20260709 13:07:34.580412 18024 raft_consensus.cc:493] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:34.580768 18024 raft_consensus.cc:3060] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:34.586220 18024 raft_consensus.cc:515] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:34.588380 18024 leader_election.cc:290] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [CANDIDATE]: Term 2 election: Requested vote from peers 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66:43325), 62de25923ada4882a388f560a3934a0b (127.7.213.67:41111)
I20260709 13:07:34.589139 18099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d" candidate_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3"
I20260709 13:07:34.589680 18174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d" candidate_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62de25923ada4882a388f560a3934a0b"
I20260709 13:07:34.589895 18099 raft_consensus.cc:3055] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:34.590278 18174 raft_consensus.cc:3060] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:34.590406 18099 raft_consensus.cc:740] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45afcea9a9b34c609e3f38bcc7f9ebe3, State: Running, Role: LEADER
I20260709 13:07:34.591354 18099 consensus_queue.cc:260] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:34.592559 18099 raft_consensus.cc:3060] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:34.596776 18174 raft_consensus.cc:2468] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e89b014bea41e5b4a48a4bdc95f53b in term 2.
I20260709 13:07:34.598243 17988 leader_election.cc:304] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49e89b014bea41e5b4a48a4bdc95f53b, 62de25923ada4882a388f560a3934a0b; no voters: 
I20260709 13:07:34.599144 18257 raft_consensus.cc:2804] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:34.599390 18099 raft_consensus.cc:2468] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e89b014bea41e5b4a48a4bdc95f53b in term 2.
I20260709 13:07:34.599604 18257 raft_consensus.cc:697] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 LEADER]: Becoming Leader. State: Replica: 49e89b014bea41e5b4a48a4bdc95f53b, State: Running, Role: LEADER
I20260709 13:07:34.600697 18257 consensus_queue.cc:237] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:34.607434 17919 catalog_manager.cc:5697] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b reported cstate change: term changed from 1 to 2, leader changed from 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66) to 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65). New cstate: current_term: 2 leader_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.617797  8021 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:07:34.619485  8021 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:07:34.619926  8021 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:07:34.634109 18024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d"
dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
mode: GRACEFUL
new_leader_uuid: "62de25923ada4882a388f560a3934a0b"
 from {username='slave'} at 127.0.0.1:50276
I20260709 13:07:34.634652 18024 raft_consensus.cc:606] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 LEADER]: Received request to transfer leadership to TS 62de25923ada4882a388f560a3934a0b
I20260709 13:07:34.635818  8021 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:07:34.784852 18099 raft_consensus.cc:1275] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 2 FOLLOWER]: Refusing update from remote peer 49e89b014bea41e5b4a48a4bdc95f53b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:34.786350 18257 consensus_queue.cc:1048] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:34.789758 18174 raft_consensus.cc:1275] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 2 FOLLOWER]: Refusing update from remote peer 49e89b014bea41e5b4a48a4bdc95f53b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:34.791308 18257 consensus_queue.cc:1048] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:35.054930 18099 raft_consensus.cc:1275] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 2 FOLLOWER]: Refusing update from remote peer 49e89b014bea41e5b4a48a4bdc95f53b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:35.056315 18270 consensus_queue.cc:1048] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:35.063721 18174 raft_consensus.cc:1275] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 2 FOLLOWER]: Refusing update from remote peer 49e89b014bea41e5b4a48a4bdc95f53b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:35.065373 18270 consensus_queue.cc:1048] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [LEADER]: Connected to new peer: Peer: permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:35.072276 18270 raft_consensus.cc:993] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b: : Instructing follower 62de25923ada4882a388f560a3934a0b to start an election
I20260709 13:07:35.072795 18257 raft_consensus.cc:1081] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 LEADER]: Signalling peer 62de25923ada4882a388f560a3934a0b to start an election
I20260709 13:07:35.074352 18174 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d"
dest_uuid: "62de25923ada4882a388f560a3934a0b"
 from {username='slave'} at 127.0.0.1:57262
I20260709 13:07:35.074883 18174 raft_consensus.cc:493] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:35.075204 18174 raft_consensus.cc:3060] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:35.080157 18174 raft_consensus.cc:515] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:35.081665 18174 leader_election.cc:290] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [CANDIDATE]: Term 3 election: Requested vote from peers 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65:46449), 45afcea9a9b34c609e3f38bcc7f9ebe3 (127.7.213.66:43325)
I20260709 13:07:35.082604 18024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d" candidate_uuid: "62de25923ada4882a388f560a3934a0b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "49e89b014bea41e5b4a48a4bdc95f53b"
I20260709 13:07:35.082949 18099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94936d8fcec34871beb274ec4e451f5d" candidate_uuid: "62de25923ada4882a388f560a3934a0b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3"
I20260709 13:07:35.083268 18024 raft_consensus.cc:3055] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:35.083598 18099 raft_consensus.cc:3060] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:35.083626 18024 raft_consensus.cc:740] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 49e89b014bea41e5b4a48a4bdc95f53b, State: Running, Role: LEADER
I20260709 13:07:35.084702 18024 consensus_queue.cc:260] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:35.085700 18024 raft_consensus.cc:3060] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:35.089766 18099 raft_consensus.cc:2468] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62de25923ada4882a388f560a3934a0b in term 3.
I20260709 13:07:35.090564 18024 raft_consensus.cc:2468] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62de25923ada4882a388f560a3934a0b in term 3.
I20260709 13:07:35.090849 18139 leader_election.cc:304] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45afcea9a9b34c609e3f38bcc7f9ebe3, 62de25923ada4882a388f560a3934a0b; no voters: 
I20260709 13:07:35.091647 18247 raft_consensus.cc:2804] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:35.092564 18247 raft_consensus.cc:697] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 3 LEADER]: Becoming Leader. State: Replica: 62de25923ada4882a388f560a3934a0b, State: Running, Role: LEADER
I20260709 13:07:35.093348 18247 consensus_queue.cc:237] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } }
I20260709 13:07:35.099552 17920 catalog_manager.cc:5697] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b reported cstate change: term changed from 2 to 3, leader changed from 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65) to 62de25923ada4882a388f560a3934a0b (127.7.213.67). New cstate: current_term: 3 leader_uuid: "62de25923ada4882a388f560a3934a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e89b014bea41e5b4a48a4bdc95f53b" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45afcea9a9b34c609e3f38bcc7f9ebe3" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62de25923ada4882a388f560a3934a0b" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 41111 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.254016  8021 tablet_server.cc:179] TabletServer@127.7.213.65:0 shutting down...
I20260709 13:07:35.271450  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:35.272184  8021 tablet_replica.cc:333] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b: stopping tablet replica
I20260709 13:07:35.272825  8021 raft_consensus.cc:2243] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.273283  8021 raft_consensus.cc:2272] T 94936d8fcec34871beb274ec4e451f5d P 49e89b014bea41e5b4a48a4bdc95f53b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.275452  8021 tablet_replica.cc:333] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b: stopping tablet replica
I20260709 13:07:35.276064  8021 raft_consensus.cc:2243] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.276855  8021 raft_consensus.cc:2272] T 78da03bd48154a5197984894e4dddf69 P 49e89b014bea41e5b4a48a4bdc95f53b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.279088  8021 tablet_replica.cc:333] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b: stopping tablet replica
I20260709 13:07:35.279621  8021 raft_consensus.cc:2243] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.280093  8021 raft_consensus.cc:2272] T e3ea749ddf6a48d8b31aefb041f177e0 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.281817  8021 tablet_replica.cc:333] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b: stopping tablet replica
I20260709 13:07:35.282277  8021 raft_consensus.cc:2243] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.283020  8021 raft_consensus.cc:2272] T 157ba30eb6b74f38a760e54abcda90f9 P 49e89b014bea41e5b4a48a4bdc95f53b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.305682  8021 tablet_server.cc:196] TabletServer@127.7.213.65:0 shutdown complete.
I20260709 13:07:35.317358  8021 tablet_server.cc:179] TabletServer@127.7.213.66:0 shutting down...
I20260709 13:07:35.336851  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:35.337477  8021 tablet_replica.cc:333] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3: stopping tablet replica
I20260709 13:07:35.338063  8021 raft_consensus.cc:2243] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.338555  8021 raft_consensus.cc:2272] T 94936d8fcec34871beb274ec4e451f5d P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.340794  8021 tablet_replica.cc:333] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3: stopping tablet replica
I20260709 13:07:35.341359  8021 raft_consensus.cc:2243] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.341801  8021 raft_consensus.cc:2272] T 78da03bd48154a5197984894e4dddf69 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.344303  8021 tablet_replica.cc:333] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3: stopping tablet replica
I20260709 13:07:35.344822  8021 raft_consensus.cc:2243] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.345233  8021 raft_consensus.cc:2272] T 157ba30eb6b74f38a760e54abcda90f9 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.346938  8021 tablet_replica.cc:333] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3: stopping tablet replica
W20260709 13:07:35.346886 18065 consensus_peers.cc:597] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 -> Peer 49e89b014bea41e5b4a48a4bdc95f53b (127.7.213.65:46449): Couldn't send request to peer 49e89b014bea41e5b4a48a4bdc95f53b. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:46449: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:35.347651  8021 raft_consensus.cc:2243] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.348454  8021 raft_consensus.cc:2272] T e3ea749ddf6a48d8b31aefb041f177e0 P 45afcea9a9b34c609e3f38bcc7f9ebe3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.370275  8021 tablet_server.cc:196] TabletServer@127.7.213.66:0 shutdown complete.
I20260709 13:07:35.381910  8021 tablet_server.cc:179] TabletServer@127.7.213.67:0 shutting down...
I20260709 13:07:35.398730  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:35.399381  8021 tablet_replica.cc:333] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b: stopping tablet replica
I20260709 13:07:35.400023  8021 raft_consensus.cc:2243] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.400686  8021 pending_rounds.cc:70] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b: Trying to abort 1 pending ops.
I20260709 13:07:35.400852  8021 pending_rounds.cc:77] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635656070066176 op_type: NO_OP noop_request { }
I20260709 13:07:35.401117  8021 raft_consensus.cc:2889] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:07:35.401347  8021 raft_consensus.cc:2272] T 94936d8fcec34871beb274ec4e451f5d P 62de25923ada4882a388f560a3934a0b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.403446  8021 tablet_replica.cc:333] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b: stopping tablet replica
I20260709 13:07:35.403985  8021 raft_consensus.cc:2243] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.404378  8021 raft_consensus.cc:2272] T 78da03bd48154a5197984894e4dddf69 P 62de25923ada4882a388f560a3934a0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.406746  8021 tablet_replica.cc:333] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b: stopping tablet replica
I20260709 13:07:35.407208  8021 raft_consensus.cc:2243] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.407573  8021 raft_consensus.cc:2272] T e3ea749ddf6a48d8b31aefb041f177e0 P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.409325  8021 tablet_replica.cc:333] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b: stopping tablet replica
I20260709 13:07:35.409726  8021 raft_consensus.cc:2243] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:35.410063  8021 raft_consensus.cc:2272] T 157ba30eb6b74f38a760e54abcda90f9 P 62de25923ada4882a388f560a3934a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.421571  8021 tablet_server.cc:196] TabletServer@127.7.213.67:0 shutdown complete.
I20260709 13:07:35.433359  8021 master.cc:562] Master@127.7.213.126:33675 shutting down...
I20260709 13:07:35.448179  8021 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:35.448750  8021 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:35.449110  8021 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6f6b715ed3524ea08c8d04c9ec470892: stopping tablet replica
I20260709 13:07:35.468336  8021 master.cc:584] Master@127.7.213.126:33675 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2997 ms)
[----------] 6 tests from LeaderRebalancerTest (297116 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:07:35.495301  8021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.213.126:34049
I20260709 13:07:35.496405  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:35.501379 18275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.501370 18274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.503899 18277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.504020  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:35.504995  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:35.505196  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:35.505370  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602455505339 us; error 0 us; skew 500 ppm
I20260709 13:07:35.505906  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:35.508261  8021 webserver.cc:533] Webserver started at http://127.7.213.126:42703/ using document root <none> and password file <none>
I20260709 13:07:35.508744  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:35.508935  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:35.509222  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:35.510313  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/master-0-root/instance:
uuid: "d0993257ed3749988dc227570cae9a8b"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-9g29"
I20260709 13:07:35.514953  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:07:35.518366 18282 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.519174  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:35.519528  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/master-0-root
uuid: "d0993257ed3749988dc227570cae9a8b"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-9g29"
I20260709 13:07:35.519939  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:35.531675  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:35.532954  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:35.568616  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.126:34049
I20260709 13:07:35.568746 18333 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.126:34049 every 8 connection(s)
I20260709 13:07:35.572327 18334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:35.585624 18334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b: Bootstrap starting.
I20260709 13:07:35.589986 18334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.593756 18334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b: No bootstrap required, opened a new log
I20260709 13:07:35.595683 18334 raft_consensus.cc:359] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0993257ed3749988dc227570cae9a8b" member_type: VOTER }
I20260709 13:07:35.596132 18334 raft_consensus.cc:385] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.596318 18334 raft_consensus.cc:740] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0993257ed3749988dc227570cae9a8b, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.596784 18334 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0993257ed3749988dc227570cae9a8b" member_type: VOTER }
I20260709 13:07:35.597187 18334 raft_consensus.cc:399] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:35.597368 18334 raft_consensus.cc:493] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:35.597584 18334 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.602404 18334 raft_consensus.cc:515] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0993257ed3749988dc227570cae9a8b" member_type: VOTER }
I20260709 13:07:35.602890 18334 leader_election.cc:304] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0993257ed3749988dc227570cae9a8b; no voters: 
I20260709 13:07:35.604002 18334 leader_election.cc:290] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:35.604362 18337 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.605806 18337 raft_consensus.cc:697] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 1 LEADER]: Becoming Leader. State: Replica: d0993257ed3749988dc227570cae9a8b, State: Running, Role: LEADER
I20260709 13:07:35.606505 18337 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0993257ed3749988dc227570cae9a8b" member_type: VOTER }
I20260709 13:07:35.606801 18334 sys_catalog.cc:565] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:35.611153 18338 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0993257ed3749988dc227570cae9a8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0993257ed3749988dc227570cae9a8b" member_type: VOTER } }
I20260709 13:07:35.611119 18339 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0993257ed3749988dc227570cae9a8b. Latest consensus state: current_term: 1 leader_uuid: "d0993257ed3749988dc227570cae9a8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0993257ed3749988dc227570cae9a8b" member_type: VOTER } }
I20260709 13:07:35.611981 18338 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:35.612023 18339 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:35.614122 18344 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:35.618808 18344 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:35.623564  8021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:35.627837 18344 catalog_manager.cc:1383] Generated new cluster ID: aac48f7567f4435ab99bc418aec765e7
I20260709 13:07:35.628124 18344 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:35.666782 18344 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:35.668123 18344 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:35.681731 18344 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b: Generated new TSK 0
I20260709 13:07:35.682345 18344 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:35.689716  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:35.695498 18355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.696522 18356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.698035 18358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.698451  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:35.699522  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:35.699784  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:35.699957  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602455699942 us; error 0 us; skew 500 ppm
I20260709 13:07:35.700474  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:35.702916  8021 webserver.cc:533] Webserver started at http://127.7.213.65:32889/ using document root <none> and password file <none>
I20260709 13:07:35.703387  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:35.703559  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:35.703862  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:35.704954  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-0-root/instance:
uuid: "45e64265fd5046af90eefab4e50526f5"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-9g29"
I20260709 13:07:35.709986  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:07:35.713619 18363 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.714452  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:07:35.714815  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "45e64265fd5046af90eefab4e50526f5"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-9g29"
I20260709 13:07:35.715094  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:35.729666  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:35.730772  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:35.732267  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:35.734669  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:35.734858  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.735174  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:35.735355  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.774732  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:46203
I20260709 13:07:35.774839 18425 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:46203 every 8 connection(s)
I20260709 13:07:35.779925  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:35.787313 18430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.789171 18431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.790340  8021 server_base.cc:1061] running on GCE node
W20260709 13:07:35.791451 18434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.791909 18426 heartbeater.cc:344] Connected to a master server at 127.7.213.126:34049
I20260709 13:07:35.792348 18426 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:35.792476  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:35.792793  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:35.792960  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602455792930 us; error 0 us; skew 500 ppm
I20260709 13:07:35.793259 18426 heartbeater.cc:507] Master 127.7.213.126:34049 requested a full tablet report, sending...
I20260709 13:07:35.793524  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:35.795432 18299 ts_manager.cc:194] Registered new tserver with Master: 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203)
I20260709 13:07:35.797163 18299 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39410
I20260709 13:07:35.797981  8021 webserver.cc:533] Webserver started at http://127.7.213.66:36011/ using document root <none> and password file <none>
I20260709 13:07:35.798512  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:35.798720  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:35.798975  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:35.800076  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-1-root/instance:
uuid: "9695202e5bb14e639ecbba6b25697dfa"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-9g29"
I20260709 13:07:35.804407  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:07:35.807541 18438 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.808370  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:07:35.808728  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-1-root
uuid: "9695202e5bb14e639ecbba6b25697dfa"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-9g29"
I20260709 13:07:35.809010  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:35.825522  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:35.826529  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:35.827960  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:35.830062  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:35.830281  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.830513  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:35.830684  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.867388  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.66:45183
I20260709 13:07:35.867498 18500 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.66:45183 every 8 connection(s)
I20260709 13:07:35.872268  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:35.878386 18504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:35.880364 18505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.882884  8021 server_base.cc:1061] running on GCE node
W20260709 13:07:35.883386 18507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:35.884608  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:35.884871  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:35.884996 18501 heartbeater.cc:344] Connected to a master server at 127.7.213.126:34049
I20260709 13:07:35.885120  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602455885100 us; error 0 us; skew 500 ppm
I20260709 13:07:35.885501 18501 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:35.885977  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:35.886390 18501 heartbeater.cc:507] Master 127.7.213.126:34049 requested a full tablet report, sending...
I20260709 13:07:35.888616  8021 webserver.cc:533] Webserver started at http://127.7.213.67:37767/ using document root <none> and password file <none>
I20260709 13:07:35.888805 18299 ts_manager.cc:194] Registered new tserver with Master: 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:35.889302  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:35.889561  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:35.889889  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:35.890638 18299 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39422
I20260709 13:07:35.891291  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-2-root/instance:
uuid: "c15cc53a71664205bbbbc284c1d299ff"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-9g29"
I20260709 13:07:35.896130  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:07:35.899225 18512 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.900086  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:07:35.900383  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-2-root
uuid: "c15cc53a71664205bbbbc284c1d299ff"
format_stamp: "Formatted at 2026-07-09 13:07:35 on dist-test-slave-9g29"
I20260709 13:07:35.900738  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602158212224-8021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:35.914570  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:35.915629  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:35.917162  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:35.919384  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:35.919607  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.919867  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:35.920046  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:35.958345  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.67:35945
I20260709 13:07:35.958447 18574 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.67:35945 every 8 connection(s)
I20260709 13:07:35.971999 18575 heartbeater.cc:344] Connected to a master server at 127.7.213.126:34049
I20260709 13:07:35.972429 18575 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:35.973150 18575 heartbeater.cc:507] Master 127.7.213.126:34049 requested a full tablet report, sending...
I20260709 13:07:35.975111 18299 ts_manager.cc:194] Registered new tserver with Master: c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:35.975258  8021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013834001s
I20260709 13:07:35.976563 18299 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39438
I20260709 13:07:36.000800 18299 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39452:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:36.003921 18299 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:36.070905 18540 tablet_service.cc:1511] Processing CreateTablet for tablet 930abb26f64e42619494dba489c18df1 (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:36.071727 18539 tablet_service.cc:1511] Processing CreateTablet for tablet ac609b21527a4972b105bdb06f4caaca (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:36.072445 18540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930abb26f64e42619494dba489c18df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.073227 18539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac609b21527a4972b105bdb06f4caaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.078588 18466 tablet_service.cc:1511] Processing CreateTablet for tablet 930abb26f64e42619494dba489c18df1 (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:36.080785 18465 tablet_service.cc:1511] Processing CreateTablet for tablet ac609b21527a4972b105bdb06f4caaca (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:36.082322 18465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac609b21527a4972b105bdb06f4caaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.084815 18464 tablet_service.cc:1511] Processing CreateTablet for tablet 0bda43ca086543e7b9665d96bdd452fa (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:36.086226 18464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bda43ca086543e7b9665d96bdd452fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.087817 18463 tablet_service.cc:1511] Processing CreateTablet for tablet a6f9c7fa6f8b4a93affd76594fe4ed6e (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:36.089252 18463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6f9c7fa6f8b4a93affd76594fe4ed6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.095335 18538 tablet_service.cc:1511] Processing CreateTablet for tablet 0bda43ca086543e7b9665d96bdd452fa (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:36.096859 18538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bda43ca086543e7b9665d96bdd452fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.096745 18535 tablet_service.cc:1511] Processing CreateTablet for tablet b2328b3b8dd64a41b451825e2156884b (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:36.098762 18535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2328b3b8dd64a41b451825e2156884b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.100795 18537 tablet_service.cc:1511] Processing CreateTablet for tablet a6f9c7fa6f8b4a93affd76594fe4ed6e (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:36.095350 18536 tablet_service.cc:1511] Processing CreateTablet for tablet b465daf7d17e4407b2226ba37042529c (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:36.104382 18537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6f9c7fa6f8b4a93affd76594fe4ed6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.108052 18462 tablet_service.cc:1511] Processing CreateTablet for tablet b465daf7d17e4407b2226ba37042529c (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:36.109093 18590 tablet_bootstrap.cc:492] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:36.109500 18462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b465daf7d17e4407b2226ba37042529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.109251 18459 tablet_service.cc:1511] Processing CreateTablet for tablet fa0f514d6ce1487ab6000263ade7054c (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:36.110631 18459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0f514d6ce1487ab6000263ade7054c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.112871 18460 tablet_service.cc:1511] Processing CreateTablet for tablet d3c63b593e614ba995254f08593ea75b (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:36.113361 18458 tablet_service.cc:1511] Processing CreateTablet for tablet 6885c0e03a4a403d9cdaad7803be4963 (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:36.114238 18460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c63b593e614ba995254f08593ea75b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.115257 18458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6885c0e03a4a403d9cdaad7803be4963. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.115403 18590 tablet_bootstrap.cc:654] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.111527 18461 tablet_service.cc:1511] Processing CreateTablet for tablet b2328b3b8dd64a41b451825e2156884b (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:36.121675 18540 tablet_service.cc:1511] Processing CreateTablet for tablet d3c63b593e614ba995254f08593ea75b (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:36.122970 18540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c63b593e614ba995254f08593ea75b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.122726 18534 tablet_service.cc:1511] Processing CreateTablet for tablet fa0f514d6ce1487ab6000263ade7054c (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:36.125814 18533 tablet_service.cc:1511] Processing CreateTablet for tablet 6885c0e03a4a403d9cdaad7803be4963 (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:36.127357 18533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6885c0e03a4a403d9cdaad7803be4963. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.129876 18391 tablet_service.cc:1511] Processing CreateTablet for tablet 930abb26f64e42619494dba489c18df1 (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:36.131343 18391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930abb26f64e42619494dba489c18df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.132047 18534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0f514d6ce1487ab6000263ade7054c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.133268 18590 tablet_bootstrap.cc:492] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:36.133850 18590 ts_tablet_manager.cc:1403] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.000s
I20260709 13:07:36.135282 18390 tablet_service.cc:1511] Processing CreateTablet for tablet ac609b21527a4972b105bdb06f4caaca (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:36.136672 18390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac609b21527a4972b105bdb06f4caaca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.137197 18590 raft_consensus.cc:359] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.138080 18590 raft_consensus.cc:385] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.138417 18590 raft_consensus.cc:740] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.139271 18590 consensus_queue.cc:260] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.141402 18389 tablet_service.cc:1511] Processing CreateTablet for tablet 0bda43ca086543e7b9665d96bdd452fa (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:36.142843 18389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bda43ca086543e7b9665d96bdd452fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.144145 18461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2328b3b8dd64a41b451825e2156884b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.148087 18466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 930abb26f64e42619494dba489c18df1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.154011 18388 tablet_service.cc:1511] Processing CreateTablet for tablet a6f9c7fa6f8b4a93affd76594fe4ed6e (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:36.156154 18388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6f9c7fa6f8b4a93affd76594fe4ed6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.156774 18536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b465daf7d17e4407b2226ba37042529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.162024 18387 tablet_service.cc:1511] Processing CreateTablet for tablet b465daf7d17e4407b2226ba37042529c (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:36.166029 18386 tablet_service.cc:1511] Processing CreateTablet for tablet b2328b3b8dd64a41b451825e2156884b (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:36.174170 18385 tablet_service.cc:1511] Processing CreateTablet for tablet d3c63b593e614ba995254f08593ea75b (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:36.176394 18387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b465daf7d17e4407b2226ba37042529c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.180048 18385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c63b593e614ba995254f08593ea75b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.184198 18386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2328b3b8dd64a41b451825e2156884b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.201508 18384 tablet_service.cc:1511] Processing CreateTablet for tablet fa0f514d6ce1487ab6000263ade7054c (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:36.203630 18384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0f514d6ce1487ab6000263ade7054c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.210016 18383 tablet_service.cc:1511] Processing CreateTablet for tablet 6885c0e03a4a403d9cdaad7803be4963 (DEFAULT_TABLE table=test-workload [id=1b7a80d6e3d54dbc9d49772ef930c915]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:36.212060 18383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6885c0e03a4a403d9cdaad7803be4963. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:36.226032 18575 heartbeater.cc:499] Master 127.7.213.126:34049 was elected leader, sending a full tablet report...
I20260709 13:07:36.252971 18590 ts_tablet_manager.cc:1434] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.119s	user 0.037s	sys 0.032s
I20260709 13:07:36.254169 18590 tablet_bootstrap.cc:492] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:36.281311 18594 tablet_bootstrap.cc:492] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:36.283514 18590 tablet_bootstrap.cc:654] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.279722 18593 tablet_bootstrap.cc:492] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:36.292260 18593 tablet_bootstrap.cc:654] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.296932 18594 tablet_bootstrap.cc:654] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.301484 18590 tablet_bootstrap.cc:492] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:36.302512 18590 ts_tablet_manager.cc:1403] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.002s
I20260709 13:07:36.305481 18593 tablet_bootstrap.cc:492] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:36.305610 18590 raft_consensus.cc:359] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.306298 18593 ts_tablet_manager.cc:1403] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.005s
I20260709 13:07:36.306594 18590 raft_consensus.cc:385] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.307148 18590 raft_consensus.cc:740] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.307958 18590 consensus_queue.cc:260] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.310029 18590 ts_tablet_manager.cc:1434] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.006s
I20260709 13:07:36.309952 18593 raft_consensus.cc:359] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.311074 18590 tablet_bootstrap.cc:492] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:36.311058 18593 raft_consensus.cc:385] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.311520 18593 raft_consensus.cc:740] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.312916 18593 consensus_queue.cc:260] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.316435 18594 tablet_bootstrap.cc:492] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:36.317143 18594 ts_tablet_manager.cc:1403] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.004s
I20260709 13:07:36.318308 18590 tablet_bootstrap.cc:654] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.320338 18594 raft_consensus.cc:359] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.321235 18594 raft_consensus.cc:385] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.321718 18594 raft_consensus.cc:740] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.322726 18594 consensus_queue.cc:260] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.325325 18501 heartbeater.cc:499] Master 127.7.213.126:34049 was elected leader, sending a full tablet report...
I20260709 13:07:36.326200 18593 ts_tablet_manager.cc:1434] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.007s
I20260709 13:07:36.327646 18593 tablet_bootstrap.cc:492] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:36.330756 18426 heartbeater.cc:499] Master 127.7.213.126:34049 was elected leader, sending a full tablet report...
I20260709 13:07:36.332028 18594 ts_tablet_manager.cc:1434] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:07:36.333324 18594 tablet_bootstrap.cc:492] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:36.340163 18594 tablet_bootstrap.cc:654] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.343612 18590 tablet_bootstrap.cc:492] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:36.344198 18590 ts_tablet_manager.cc:1403] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.018s
I20260709 13:07:36.346306 18594 tablet_bootstrap.cc:492] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:36.346462 18590 raft_consensus.cc:359] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.347004 18594 ts_tablet_manager.cc:1403] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:07:36.347261 18590 raft_consensus.cc:385] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.347615 18590 raft_consensus.cc:740] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.348402 18593 tablet_bootstrap.cc:654] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.348534 18590 consensus_queue.cc:260] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.350250 18594 raft_consensus.cc:359] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.351126 18594 raft_consensus.cc:385] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.351580 18594 raft_consensus.cc:740] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.353333 18590 ts_tablet_manager.cc:1434] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:07:36.352651 18594 consensus_queue.cc:260] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.354480 18590 tablet_bootstrap.cc:492] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:36.356261 18594 ts_tablet_manager.cc:1434] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.003s
I20260709 13:07:36.357733 18594 tablet_bootstrap.cc:492] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:36.361079 18590 tablet_bootstrap.cc:654] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.366724 18594 tablet_bootstrap.cc:654] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.373248 18590 tablet_bootstrap.cc:492] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:36.373755 18590 ts_tablet_manager.cc:1403] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.008s
I20260709 13:07:36.376760 18592 raft_consensus.cc:493] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.377249 18592 raft_consensus.cc:515] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.393368 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.394863 18593 tablet_bootstrap.cc:492] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:36.395609 18593 ts_tablet_manager.cc:1403] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.068s	user 0.039s	sys 0.014s
W20260709 13:07:36.396451 18513 leader_election.cc:343] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.394639 18592 leader_election.cc:290] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:36.397522 18590 raft_consensus.cc:359] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.399225 18590 raft_consensus.cc:385] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.398914 18593 raft_consensus.cc:359] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.399756 18590 raft_consensus.cc:740] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.399964 18593 raft_consensus.cc:385] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.400431 18593 raft_consensus.cc:740] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.401338 18593 consensus_queue.cc:260] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.400990 18590 consensus_queue.cc:260] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.403649 18593 ts_tablet_manager.cc:1434] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:36.404697 18593 tablet_bootstrap.cc:492] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:36.405233 18600 raft_consensus.cc:493] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.405740 18600 raft_consensus.cc:515] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.411129 18593 tablet_bootstrap.cc:654] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.414880 18600 leader_election.cc:290] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.425565 18590 ts_tablet_manager.cc:1434] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.051s	user 0.009s	sys 0.001s
I20260709 13:07:36.436847 18590 tablet_bootstrap.cc:492] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:36.443130 18594 tablet_bootstrap.cc:492] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:36.443828 18594 ts_tablet_manager.cc:1403] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.086s	user 0.034s	sys 0.045s
I20260709 13:07:36.447233 18594 raft_consensus.cc:359] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.448148 18594 raft_consensus.cc:385] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.447855 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:36.448589 18594 raft_consensus.cc:740] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.448971 18476 raft_consensus.cc:2468] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:36.449553 18594 consensus_queue.cc:260] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.450732 18516 leader_election.cc:304] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:36.452148 18594 ts_tablet_manager.cc:1434] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:36.452468 18592 raft_consensus.cc:2804] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.452919 18592 raft_consensus.cc:493] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.453248 18592 raft_consensus.cc:3060] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.453301 18594 tablet_bootstrap.cc:492] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:36.460475 18594 tablet_bootstrap.cc:654] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.447233 18590 tablet_bootstrap.cc:654] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.469681 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
W20260709 13:07:36.471267 18367 leader_election.cc:343] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.463394 18592 raft_consensus.cc:515] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.473850 18592 leader_election.cc:290] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:36.475118 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:36.477828 18401 raft_consensus.cc:3060] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.479141 18593 tablet_bootstrap.cc:492] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:36.479766 18593 ts_tablet_manager.cc:1403] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.075s	user 0.031s	sys 0.025s
I20260709 13:07:36.475540 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:36.480367 18476 raft_consensus.cc:3060] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.480532 18594 tablet_bootstrap.cc:492] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:36.481148 18594 ts_tablet_manager.cc:1403] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.012s
I20260709 13:07:36.481799 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
W20260709 13:07:36.483714 18365 leader_election.cc:343] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.483557 18593 raft_consensus.cc:359] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.484532 18593 raft_consensus.cc:385] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.484297 18594 raft_consensus.cc:359] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.485013 18593 raft_consensus.cc:740] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.485415 18594 raft_consensus.cc:385] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.485826 18594 raft_consensus.cc:740] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.485937 18593 consensus_queue.cc:260] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.486883 18594 consensus_queue.cc:260] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.477089 18590 tablet_bootstrap.cc:492] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:36.488145 18590 ts_tablet_manager.cc:1403] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.052s	user 0.008s	sys 0.005s
I20260709 13:07:36.488574 18593 ts_tablet_manager.cc:1434] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 13:07:36.484311 18365 leader_election.cc:304] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5; no voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:36.489542 18594 ts_tablet_manager.cc:1434] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:36.490621 18594 tablet_bootstrap.cc:492] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:36.491861 18590 raft_consensus.cc:359] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.489480 18600 raft_consensus.cc:2749] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.492632 18590 raft_consensus.cc:385] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.493827 18590 raft_consensus.cc:740] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.494809 18590 consensus_queue.cc:260] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.496905 18401 raft_consensus.cc:2468] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:36.497481 18590 ts_tablet_manager.cc:1434] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:07:36.498111 18594 tablet_bootstrap.cc:654] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.490658 18593 tablet_bootstrap.cc:492] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:36.503238 18476 raft_consensus.cc:2468] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:36.505038 18516 leader_election.cc:304] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:36.506237 18592 raft_consensus.cc:2804] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.512147 18590 tablet_bootstrap.cc:492] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:36.515079 18593 tablet_bootstrap.cc:654] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.525328 18592 raft_consensus.cc:697] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:36.526866 18592 consensus_queue.cc:237] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.528129 18590 tablet_bootstrap.cc:654] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.534446 18613 raft_consensus.cc:493] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.535180 18613 raft_consensus.cc:515] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.540586 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.541183 18594 tablet_bootstrap.cc:492] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:36.541476 18401 raft_consensus.cc:2468] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:36.542026 18594 ts_tablet_manager.cc:1403] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.052s	user 0.022s	sys 0.023s
I20260709 13:07:36.543520 18513 leader_election.cc:304] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:36.547725 18593 tablet_bootstrap.cc:492] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:36.547367 18594 raft_consensus.cc:359] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.548664 18594 raft_consensus.cc:385] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.548821 18593 ts_tablet_manager.cc:1403] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.019s
I20260709 13:07:36.549153 18594 raft_consensus.cc:740] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.550350 18594 consensus_queue.cc:260] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.551836 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:36.552392 18613 leader_election.cc:290] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:36.553006 18613 raft_consensus.cc:2804] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.554200 18613 raft_consensus.cc:493] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.553629 18594 ts_tablet_manager.cc:1434] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.000s
I20260709 13:07:36.555968 18594 tablet_bootstrap.cc:492] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:36.557459 18590 tablet_bootstrap.cc:492] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:36.554719 18613 raft_consensus.cc:3060] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.554703 18593 raft_consensus.cc:359] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.559156 18593 raft_consensus.cc:385] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.558071 18590 ts_tablet_manager.cc:1403] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.023s
I20260709 13:07:36.559556 18593 raft_consensus.cc:740] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
W20260709 13:07:36.563194 18516 leader_election.cc:343] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.564038 18593 consensus_queue.cc:260] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.565565 18594 tablet_bootstrap.cc:654] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.570204 18593 ts_tablet_manager.cc:1434] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 13:07:36.571269 18593 tablet_bootstrap.cc:492] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:36.577891 18298 catalog_manager.cc:5697] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 0 to 1, leader changed from <none> to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 1 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.568439 18590 raft_consensus.cc:359] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.580207 18590 raft_consensus.cc:385] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.580394 18598 raft_consensus.cc:493] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.584277 18590 raft_consensus.cc:740] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.584474 18598 raft_consensus.cc:515] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.585237 18590 consensus_queue.cc:260] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.585783 18613 raft_consensus.cc:515] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.587894 18590 ts_tablet_manager.cc:1434] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:07:36.588155 18594 tablet_bootstrap.cc:492] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:36.588702 18594 ts_tablet_manager.cc:1403] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.018s
I20260709 13:07:36.589005 18590 tablet_bootstrap.cc:492] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:36.589458 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:36.590103 18401 raft_consensus.cc:3060] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.592234 18594 raft_consensus.cc:359] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.593187 18594 raft_consensus.cc:385] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.593556 18594 raft_consensus.cc:740] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.594615 18594 consensus_queue.cc:260] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.595942 18598 leader_election.cc:290] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945), 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203)
I20260709 13:07:36.597160 18590 tablet_bootstrap.cc:654] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.597412 18594 ts_tablet_manager.cc:1434] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:07:36.598690 18594 tablet_bootstrap.cc:492] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:36.603132 18613 leader_election.cc:290] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:36.605005 18594 tablet_bootstrap.cc:654] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.586298 18593 tablet_bootstrap.cc:654] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.601998 18401 raft_consensus.cc:2468] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:36.609571 18590 tablet_bootstrap.cc:492] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:36.609767 18513 leader_election.cc:304] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:36.610121 18590 ts_tablet_manager.cc:1403] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 13:07:36.604852 18613 raft_consensus.cc:493] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.613765 18598 raft_consensus.cc:493] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.612103 18613 raft_consensus.cc:515] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.614336 18598 raft_consensus.cc:515] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.617992 18590 raft_consensus.cc:359] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.618831 18590 raft_consensus.cc:385] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.619146 18590 raft_consensus.cc:740] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
W20260709 13:07:36.619688 18347 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:36.620007 18590 consensus_queue.cc:260] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.621834 18598 leader_election.cc:290] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.621412 18592 raft_consensus.cc:2804] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.628366 18590 ts_tablet_manager.cc:1434] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.003s
I20260709 13:07:36.628650 18613 leader_election.cc:290] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203)
I20260709 13:07:36.630831 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.629789 18590 tablet_bootstrap.cc:492] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:36.631841 18401 raft_consensus.cc:2468] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:36.635354 18513 leader_election.cc:304] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:36.629956 18592 raft_consensus.cc:697] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:36.638907 18592 consensus_queue.cc:237] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.643762 18617 raft_consensus.cc:2804] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.645969 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:36.646731 18593 tablet_bootstrap.cc:492] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:36.647518 18593 ts_tablet_manager.cc:1403] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.076s	user 0.010s	sys 0.005s
I20260709 13:07:36.647361 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:36.648350 18549 raft_consensus.cc:2468] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:36.650385 18440 leader_election.cc:304] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:36.651144 18593 raft_consensus.cc:359] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.651647 18598 raft_consensus.cc:2804] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.652192 18593 raft_consensus.cc:385] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.652274 18598 raft_consensus.cc:493] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.652829 18593 raft_consensus.cc:740] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.652994 18598 raft_consensus.cc:3060] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.654042 18593 consensus_queue.cc:260] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.657112 18625 raft_consensus.cc:493] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.657758 18625 raft_consensus.cc:515] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.658202 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:36.659530 18594 tablet_bootstrap.cc:492] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:36.659933 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:36.660172 18594 ts_tablet_manager.cc:1403] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.027s
I20260709 13:07:36.660701 18476 raft_consensus.cc:2468] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:36.661115 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c63b593e614ba995254f08593ea75b" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
W20260709 13:07:36.661876 18516 leader_election.cc:343] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:36.662693 18513 leader_election.cc:343] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.663875 18594 raft_consensus.cc:359] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.664687 18594 raft_consensus.cc:385] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.665060 18594 raft_consensus.cc:740] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.664966 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c63b593e614ba995254f08593ea75b" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:36.665716 18590 tablet_bootstrap.cc:654] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.666105 18617 raft_consensus.cc:493] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.666772 18594 consensus_queue.cc:260] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.667497 18617 raft_consensus.cc:3060] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:36.668042 18516 leader_election.cc:343] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.672050 18516 leader_election.cc:304] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:36.672871 18613 raft_consensus.cc:2749] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.663888 18625 leader_election.cc:290] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:36.675328 18594 ts_tablet_manager.cc:1434] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.005s
I20260709 13:07:36.676151 18593 ts_tablet_manager.cc:1434] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.000s
I20260709 13:07:36.676527 18594 tablet_bootstrap.cc:492] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:36.677964 18593 tablet_bootstrap.cc:492] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:36.678699 18598 raft_consensus.cc:515] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.679023 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.680097 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.680948 18401 raft_consensus.cc:2468] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
W20260709 13:07:36.682734 18439 leader_election.cc:343] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.686182 18439 leader_election.cc:304] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:36.687193 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:36.688570 18594 tablet_bootstrap.cc:654] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.687536 18628 raft_consensus.cc:2804] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.687672 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:36.692902 18549 raft_consensus.cc:3060] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.693109 18593 tablet_bootstrap.cc:654] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:36.694541 18439 leader_election.cc:343] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.691365 18590 tablet_bootstrap.cc:492] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:36.692946 18628 raft_consensus.cc:493] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.695652 18590 ts_tablet_manager.cc:1403] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.066s	user 0.018s	sys 0.001s
I20260709 13:07:36.695542 18598 leader_election.cc:290] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.701802 18617 raft_consensus.cc:515] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.703657 18550 raft_consensus.cc:2393] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9695202e5bb14e639ecbba6b25697dfa in current term 1: Already voted for candidate c15cc53a71664205bbbbc284c1d299ff in this term.
I20260709 13:07:36.704721 18617 leader_election.cc:290] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203)
I20260709 13:07:36.705483 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:36.705967 18549 raft_consensus.cc:2468] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:36.702698 18628 raft_consensus.cc:3060] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.703096 18590 raft_consensus.cc:359] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.707337 18590 raft_consensus.cc:385] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.707269 18440 leader_election.cc:304] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:36.707828 18590 raft_consensus.cc:740] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.708777 18590 consensus_queue.cc:260] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.709180 18598 raft_consensus.cc:2804] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.711627 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:36.712293 18401 raft_consensus.cc:3060] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.712504 18590 ts_tablet_manager.cc:1434] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:07:36.713999 18590 tablet_bootstrap.cc:492] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:36.714795 18592 raft_consensus.cc:493] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.715302 18592 raft_consensus.cc:515] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.718004 18592 leader_election.cc:290] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:36.719676 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0f514d6ce1487ab6000263ade7054c" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.720144 18475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0f514d6ce1487ab6000263ade7054c" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:36.720489 18400 raft_consensus.cc:2468] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:36.720881 18475 raft_consensus.cc:2468] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:36.722074 18590 tablet_bootstrap.cc:654] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.722026 18513 leader_election.cc:304] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:36.723939 18592 raft_consensus.cc:2804] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.724442 18598 raft_consensus.cc:697] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:36.724558 18592 raft_consensus.cc:493] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.725282 18592 raft_consensus.cc:3060] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.725630 18598 consensus_queue.cc:237] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.736191 18628 raft_consensus.cc:515] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.737399 18476 raft_consensus.cc:2393] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c15cc53a71664205bbbbc284c1d299ff in current term 1: Already voted for candidate 9695202e5bb14e639ecbba6b25697dfa in this term.
I20260709 13:07:36.735267 18592 raft_consensus.cc:515] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.741252 18592 leader_election.cc:290] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:36.742352 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0f514d6ce1487ab6000263ade7054c" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:36.743119 18476 raft_consensus.cc:3060] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.745051 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0f514d6ce1487ab6000263ade7054c" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:36.745436 18594 tablet_bootstrap.cc:492] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:36.745659 18400 raft_consensus.cc:3060] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.746196 18594 ts_tablet_manager.cc:1403] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.070s	user 0.025s	sys 0.008s
I20260709 13:07:36.749442 18594 raft_consensus.cc:359] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.750248 18594 raft_consensus.cc:385] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.750622 18594 raft_consensus.cc:740] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.750691 18476 raft_consensus.cc:2468] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:36.729389 18401 raft_consensus.cc:2468] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:36.751930 18516 leader_election.cc:304] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:36.751659 18594 consensus_queue.cc:260] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.753587 18513 leader_election.cc:304] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:36.754617 18617 raft_consensus.cc:2804] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.755189 18617 raft_consensus.cc:697] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:36.756251 18592 raft_consensus.cc:2804] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.758555 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:36.758790 18592 raft_consensus.cc:697] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:36.759449 18401 raft_consensus.cc:2393] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9695202e5bb14e639ecbba6b25697dfa in current term 1: Already voted for candidate c15cc53a71664205bbbbc284c1d299ff in this term.
I20260709 13:07:36.756036 18617 consensus_queue.cc:237] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.760416 18590 tablet_bootstrap.cc:492] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:36.755573 18298 catalog_manager.cc:5697] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 0 to 1, leader changed from <none> to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 1 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.760023 18592 consensus_queue.cc:237] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:36.757828 18628 leader_election.cc:290] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Requested vote from peers c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945), 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203)
I20260709 13:07:36.758152 18594 ts_tablet_manager.cc:1434] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:07:36.760951 18590 ts_tablet_manager.cc:1403] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.008s
I20260709 13:07:36.762509 18594 tablet_bootstrap.cc:492] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:36.764603 18590 raft_consensus.cc:359] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.765348 18590 raft_consensus.cc:385] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.765658 18590 raft_consensus.cc:740] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.766526 18590 consensus_queue.cc:260] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.769236 18590 ts_tablet_manager.cc:1434] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:36.771715 18594 tablet_bootstrap.cc:654] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.754370 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:36.777907 18440 leader_election.cc:304] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa; no voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:36.779520 18628 raft_consensus.cc:2749] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.781435 18594 tablet_bootstrap.cc:492] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:36.781997 18594 ts_tablet_manager.cc:1403] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.001s
I20260709 13:07:36.783258 18400 raft_consensus.cc:2468] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:36.785318 18594 raft_consensus.cc:359] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.786120 18594 raft_consensus.cc:385] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.786396 18594 raft_consensus.cc:740] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.786499 18593 tablet_bootstrap.cc:492] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:36.787177 18593 ts_tablet_manager.cc:1403] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.110s	user 0.013s	sys 0.007s
I20260709 13:07:36.795394 18593 raft_consensus.cc:359] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.796285 18593 raft_consensus.cc:385] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.796597 18593 raft_consensus.cc:740] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.797369 18593 consensus_queue.cc:260] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.797519 18298 catalog_manager.cc:5697] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:36.799485 18593 ts_tablet_manager.cc:1434] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.003s
I20260709 13:07:36.800426 18593 tablet_bootstrap.cc:492] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:36.799695 18299 catalog_manager.cc:5697] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 0 to 1, leader changed from <none> to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 1 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.801360 18299 catalog_manager.cc:5697] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 0 to 1, leader changed from <none> to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 1 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:36.803282 18594 consensus_queue.cc:260] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.805718 18594 ts_tablet_manager.cc:1434] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:07:36.809511 18593 tablet_bootstrap.cc:654] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.818922 18593 tablet_bootstrap.cc:492] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:36.819447 18593 ts_tablet_manager.cc:1403] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.009s
I20260709 13:07:36.822211 18593 raft_consensus.cc:359] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.822822 18593 raft_consensus.cc:385] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.823096 18593 raft_consensus.cc:740] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.823702 18593 consensus_queue.cc:260] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.825907 18593 ts_tablet_manager.cc:1434] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:36.826927 18593 tablet_bootstrap.cc:492] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:36.832353 18593 tablet_bootstrap.cc:654] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.836789 18593 tablet_bootstrap.cc:492] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:36.837157 18593 ts_tablet_manager.cc:1403] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:07:36.839088 18593 raft_consensus.cc:359] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.839567 18593 raft_consensus.cc:385] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.839803 18593 raft_consensus.cc:740] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.840314 18593 consensus_queue.cc:260] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.841747 18593 ts_tablet_manager.cc:1434] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:07:36.842767 18593 tablet_bootstrap.cc:492] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:36.848019 18593 tablet_bootstrap.cc:654] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.852418 18593 tablet_bootstrap.cc:492] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:36.852897 18593 ts_tablet_manager.cc:1403] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:07:36.854841 18593 raft_consensus.cc:359] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.855360 18593 raft_consensus.cc:385] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.855597 18593 raft_consensus.cc:740] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.856194 18593 consensus_queue.cc:260] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.857685 18593 ts_tablet_manager.cc:1434] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:36.904825 18611 raft_consensus.cc:493] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.905560 18611 raft_consensus.cc:515] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.907544 18611 leader_election.cc:290] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.908264 18611 raft_consensus.cc:493] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.908500 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b465daf7d17e4407b2226ba37042529c" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:36.908805 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b465daf7d17e4407b2226ba37042529c" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:36.909322 18476 raft_consensus.cc:2468] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 0.
I20260709 13:07:36.908783 18611 raft_consensus.cc:515] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.909658 18549 raft_consensus.cc:2468] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 0.
I20260709 13:07:36.910596 18367 leader_election.cc:304] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:36.911492 18600 raft_consensus.cc:2804] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.911502 18611 leader_election.cc:290] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.912009 18600 raft_consensus.cc:493] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.912412 18600 raft_consensus.cc:3060] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.912590 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:36.912858 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:36.913422 18549 raft_consensus.cc:2393] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45e64265fd5046af90eefab4e50526f5 in current term 1: Already voted for candidate 9695202e5bb14e639ecbba6b25697dfa in this term.
I20260709 13:07:36.914570 18365 leader_election.cc:304] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5; no voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:36.915270 18611 raft_consensus.cc:3060] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.919617 18600 raft_consensus.cc:515] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.921893 18600 leader_election.cc:290] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.922535 18611 raft_consensus.cc:2749] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.922874 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b465daf7d17e4407b2226ba37042529c" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:36.923182 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b465daf7d17e4407b2226ba37042529c" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:36.923481 18476 raft_consensus.cc:3060] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.923897 18549 raft_consensus.cc:3060] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.928306 18476 raft_consensus.cc:2468] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 1.
I20260709 13:07:36.929175 18549 raft_consensus.cc:2468] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 1.
I20260709 13:07:36.929544 18367 leader_election.cc:304] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:36.930718 18611 raft_consensus.cc:2804] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.931006 18617 raft_consensus.cc:493] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.931301 18611 raft_consensus.cc:697] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:36.931618 18617 raft_consensus.cc:515] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.932528 18598 raft_consensus.cc:493] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.932402 18611 consensus_queue.cc:237] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.933001 18598 raft_consensus.cc:515] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.933640 18617 leader_election.cc:290] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:36.934643 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.935060 18598 leader_election.cc:290] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.935386 18400 raft_consensus.cc:2468] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:36.936549 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.936800 18513 leader_election.cc:304] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:36.936980 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:36.937654 18401 raft_consensus.cc:2393] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9695202e5bb14e639ecbba6b25697dfa in current term 1: Already voted for candidate c15cc53a71664205bbbbc284c1d299ff in this term.
I20260709 13:07:36.937834 18617 raft_consensus.cc:2804] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.938352 18617 raft_consensus.cc:493] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.938758 18617 raft_consensus.cc:3060] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.939119 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:36.940317 18439 leader_election.cc:304] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa; no voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:36.941344 18598 raft_consensus.cc:3060] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.940023 18476 raft_consensus.cc:2468] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:36.948993 18299 catalog_manager.cc:5697] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:36.949443 18598 raft_consensus.cc:2749] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.949443 18617 raft_consensus.cc:515] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.951897 18617 leader_election.cc:290] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:36.952957 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:36.952957 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:36.953883 18476 raft_consensus.cc:3060] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.953924 18401 raft_consensus.cc:3060] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.960546 18476 raft_consensus.cc:2468] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:36.960582 18401 raft_consensus.cc:2468] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:36.961704 18516 leader_election.cc:304] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:36.962417 18617 raft_consensus.cc:2804] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.962917 18617 raft_consensus.cc:697] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:36.963627 18617 consensus_queue.cc:237] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.967033 18598 raft_consensus.cc:493] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.967574 18598 raft_consensus.cc:515] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.969489 18598 leader_election.cc:290] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.970932 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6885c0e03a4a403d9cdaad7803be4963" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.971603 18401 raft_consensus.cc:2468] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:36.971724 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6885c0e03a4a403d9cdaad7803be4963" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:36.972766 18439 leader_election.cc:304] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:36.972829 18549 raft_consensus.cc:2468] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:36.973623 18598 raft_consensus.cc:2804] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.972990 18299 catalog_manager.cc:5697] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 0 to 1, leader changed from <none> to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 1 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.974021 18598 raft_consensus.cc:493] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.974406 18598 raft_consensus.cc:3060] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.980679 18598 raft_consensus.cc:515] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.982775 18598 leader_election.cc:290] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.983347 18617 consensus_queue.cc:1048] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:36.983613 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6885c0e03a4a403d9cdaad7803be4963" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:36.983880 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6885c0e03a4a403d9cdaad7803be4963" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:36.984261 18401 raft_consensus.cc:3060] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.984511 18549 raft_consensus.cc:3060] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.988893 18598 raft_consensus.cc:493] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.989421 18598 raft_consensus.cc:515] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.990988 18401 raft_consensus.cc:2468] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:36.991520 18598 leader_election.cc:290] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:36.992569 18439 leader_election.cc:304] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:36.992635 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c63b593e614ba995254f08593ea75b" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:36.992990 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c63b593e614ba995254f08593ea75b" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:36.993505 18550 raft_consensus.cc:2468] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:36.993607 18598 raft_consensus.cc:2804] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.993614 18401 raft_consensus.cc:2468] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:36.994232 18549 raft_consensus.cc:2468] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:36.994340 18598 raft_consensus.cc:697] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:36.994943 18439 leader_election.cc:304] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:36.995536 18598 consensus_queue.cc:237] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:36.995922 18628 raft_consensus.cc:2804] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.996799 18628 raft_consensus.cc:493] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.997170 18628 raft_consensus.cc:3060] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.005048 18628 raft_consensus.cc:515] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.008636 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c63b593e614ba995254f08593ea75b" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:37.008971 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c63b593e614ba995254f08593ea75b" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:37.009181 18628 leader_election.cc:290] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:37.009625 18401 raft_consensus.cc:3060] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.007098 18299 catalog_manager.cc:5697] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.009346 18549 raft_consensus.cc:3060] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.012594 18592 consensus_queue.cc:1048] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.019090 18401 raft_consensus.cc:2468] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:37.020596 18439 leader_election.cc:304] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:37.021670 18628 raft_consensus.cc:2804] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.022166 18628 raft_consensus.cc:697] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:37.019593 18549 raft_consensus.cc:2468] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:37.022976 18628 consensus_queue.cc:237] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.034222 18298 catalog_manager.cc:5697] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.102619 18592 consensus_queue.cc:1048] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.114012 18617 consensus_queue.cc:1048] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.138625 18628 consensus_queue.cc:1048] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.149179 18631 consensus_queue.cc:1048] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.250854 18617 consensus_queue.cc:1048] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.264394 18592 consensus_queue.cc:1048] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.265377 18549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "930abb26f64e42619494dba489c18df1"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
mode: GRACEFUL
new_leader_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:07:37.266068 18549 raft_consensus.cc:606] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Received request to transfer leadership to TS 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:37.273135 18549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
mode: GRACEFUL
new_leader_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:07:37.273815 18549 raft_consensus.cc:606] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Received request to transfer leadership to TS 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:37.279240 18549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
mode: GRACEFUL
new_leader_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:07:37.279922 18549 raft_consensus.cc:606] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Received request to transfer leadership to TS 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:37.283716 18592 consensus_queue.cc:1048] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.294426 18625 raft_consensus.cc:993] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff: : Instructing follower 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:37.294924 18625 raft_consensus.cc:1081] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Signalling peer 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:37.296998 18401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "930abb26f64e42619494dba489c18df1"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:48596
I20260709 13:07:37.297686 18401 raft_consensus.cc:493] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.298111 18401 raft_consensus.cc:3060] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.305078 18592 consensus_queue.cc:1048] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.307111 18401 raft_consensus.cc:515] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:37.308808 18400 raft_consensus.cc:1240] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Rejecting Update request from peer c15cc53a71664205bbbbc284c1d299ff for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:37.310917 18475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:37.312835 18592 consensus_queue.cc:1059] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.313294 18401 leader_election.cc:290] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:37.313917 18625 raft_consensus.cc:3055] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.314414 18625 raft_consensus.cc:740] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:37.314445 18475 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
mode: GRACEFUL
new_leader_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:58568
I20260709 13:07:37.315138 18475 raft_consensus.cc:606] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Received request to transfer leadership to TS 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:37.314508 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "930abb26f64e42619494dba489c18df1" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:37.315382 18625 consensus_queue.cc:260] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:37.316661 18625 raft_consensus.cc:3060] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.330971 18549 raft_consensus.cc:2468] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:37.332791 18365 leader_election.cc:304] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:37.333500 18611 raft_consensus.cc:2804] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.333984 18611 raft_consensus.cc:697] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:37.334791 18611 consensus_queue.cc:237] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } }
I20260709 13:07:37.335191 18401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b465daf7d17e4407b2226ba37042529c"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
mode: GRACEFUL
new_leader_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:48610
I20260709 13:07:37.335850 18401 raft_consensus.cc:606] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Received request to transfer leadership to TS 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:37.341439 18549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
mode: GRACEFUL
new_leader_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:07:37.341943 18549 raft_consensus.cc:606] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Received request to transfer leadership to TS 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:37.342403 18298 catalog_manager.cc:5697] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 1 to 2, leader changed from c15cc53a71664205bbbbc284c1d299ff (127.7.213.67) to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:37.350369 18549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa0f514d6ce1487ab6000263ade7054c"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
mode: GRACEFUL
new_leader_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:07:37.350994 18549 raft_consensus.cc:606] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Received request to transfer leadership to TS 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:37.356253 18476 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6885c0e03a4a403d9cdaad7803be4963"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
mode: GRACEFUL
new_leader_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:58568
I20260709 13:07:37.356873 18476 raft_consensus.cc:606] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Received request to transfer leadership to TS c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:37.393275 18598 consensus_queue.cc:1048] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.399848 18631 raft_consensus.cc:993] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa: : Instructing follower c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:37.400358 18631 raft_consensus.cc:1081] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Signalling peer c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:37.402204 18550 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6885c0e03a4a403d9cdaad7803be4963"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:56938
I20260709 13:07:37.402837 18550 raft_consensus.cc:493] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.403265 18550 raft_consensus.cc:3060] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.405593 18628 consensus_queue.cc:1048] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.405790 18625 consensus_queue.cc:1048] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.411667 18550 raft_consensus.cc:515] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.413173 18549 raft_consensus.cc:1240] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Rejecting Update request from peer 9695202e5bb14e639ecbba6b25697dfa for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:37.415783 18628 consensus_queue.cc:1059] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.416683 18628 raft_consensus.cc:3055] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.417099 18628 raft_consensus.cc:740] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:37.417522 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6885c0e03a4a403d9cdaad7803be4963" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:37.418205 18401 raft_consensus.cc:3060] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.418238 18628 consensus_queue.cc:260] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.419360 18550 leader_election.cc:290] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 2 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:37.420374 18475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6885c0e03a4a403d9cdaad7803be4963" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:37.420187 18628 raft_consensus.cc:3060] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.426483 18592 consensus_queue.cc:1048] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.428598 18611 consensus_queue.cc:1048] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.426458 18401 raft_consensus.cc:2468] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 2.
I20260709 13:07:37.434084 18513 leader_election.cc:304] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:37.435117 18592 raft_consensus.cc:2804] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.435674 18592 raft_consensus.cc:697] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 2 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:37.436609 18592 consensus_queue.cc:237] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.445410 18592 raft_consensus.cc:993] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff: : Instructing follower 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:37.445904 18592 raft_consensus.cc:1081] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Signalling peer 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:37.446537 18475 raft_consensus.cc:2468] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 2.
I20260709 13:07:37.446285 18298 catalog_manager.cc:5697] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 1 to 2, leader changed from 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66) to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 2 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:37.447948 18401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:48596
I20260709 13:07:37.448698 18401 raft_consensus.cc:493] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.449096 18401 raft_consensus.cc:3060] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.457315 18401 raft_consensus.cc:515] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.458817 18600 raft_consensus.cc:993] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5: : Instructing follower 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:37.459364 18600 raft_consensus.cc:1081] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Signalling peer 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:37.461422 18401 leader_election.cc:290] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:37.462910 18475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b465daf7d17e4407b2226ba37042529c"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:58560
I20260709 13:07:37.463022 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:37.463716 18476 raft_consensus.cc:3060] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.463979 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bda43ca086543e7b9665d96bdd452fa" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:37.466013 18600 consensus_queue.cc:1048] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.466544 18550 raft_consensus.cc:3055] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.468124 18550 raft_consensus.cc:740] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:37.469197 18475 raft_consensus.cc:493] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.469599 18475 raft_consensus.cc:3060] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.473191 18476 raft_consensus.cc:2468] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:37.469204 18550 consensus_queue.cc:260] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.474803 18367 leader_election.cc:304] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:37.475848 18550 raft_consensus.cc:3060] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.475960 18600 raft_consensus.cc:2804] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.476776 18600 raft_consensus.cc:697] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:37.478003 18600 consensus_queue.cc:237] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.481807 18475 raft_consensus.cc:515] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.483801 18550 raft_consensus.cc:2468] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:37.484925 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b465daf7d17e4407b2226ba37042529c" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:37.485567 18401 raft_consensus.cc:3055] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.485661 18475 leader_election.cc:290] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:37.485924 18401 raft_consensus.cc:740] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:37.486773 18401 consensus_queue.cc:260] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.487892 18401 raft_consensus.cc:3060] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.487851 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b465daf7d17e4407b2226ba37042529c" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:37.494675 18401 raft_consensus.cc:2468] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:37.494675 18298 catalog_manager.cc:5697] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 1 to 2, leader changed from c15cc53a71664205bbbbc284c1d299ff (127.7.213.67) to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.496035 18439 leader_election.cc:304] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:37.496846 18628 raft_consensus.cc:2804] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.497383 18628 raft_consensus.cc:697] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:37.498353 18628 consensus_queue.cc:237] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.506372 18298 catalog_manager.cc:5697] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 1 to 2, leader changed from 45e64265fd5046af90eefab4e50526f5 (127.7.213.65) to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.514027 18628 consensus_queue.cc:1048] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.526098 18592 raft_consensus.cc:993] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff: : Instructing follower 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:37.526638 18592 raft_consensus.cc:1081] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Signalling peer 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:37.526511 18628 consensus_queue.cc:1048] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.528746 18475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:58550
I20260709 13:07:37.529328 18475 raft_consensus.cc:493] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.529724 18475 raft_consensus.cc:3060] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.537930 18475 raft_consensus.cc:515] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.543357 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:37.542968 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:37.544183 18550 raft_consensus.cc:3055] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.544564 18401 raft_consensus.cc:3060] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.544606 18550 raft_consensus.cc:740] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:37.546103 18475 leader_election.cc:290] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:37.545763 18550 consensus_queue.cc:260] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.547907 18550 raft_consensus.cc:3060] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.552809 18401 raft_consensus.cc:2468] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:37.554239 18439 leader_election.cc:304] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:37.555114 18628 raft_consensus.cc:2804] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.555636 18628 raft_consensus.cc:697] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:37.555845 18550 raft_consensus.cc:2468] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:37.556511 18628 consensus_queue.cc:237] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.563185 18298 catalog_manager.cc:5697] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 1 to 2, leader changed from c15cc53a71664205bbbbc284c1d299ff (127.7.213.67) to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.670307 18617 raft_consensus.cc:993] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff: : Instructing follower 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:37.670778 18592 raft_consensus.cc:1081] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Signalling peer 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:37.672327 18401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:48596
I20260709 13:07:37.672962 18401 raft_consensus.cc:493] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.673362 18401 raft_consensus.cc:3060] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.678818 18401 raft_consensus.cc:515] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.681010 18401 leader_election.cc:290] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:37.681926 18475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:37.682137 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac609b21527a4972b105bdb06f4caaca" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:37.682555 18475 raft_consensus.cc:3060] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.682807 18550 raft_consensus.cc:3055] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.683192 18550 raft_consensus.cc:740] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:37.683952 18550 consensus_queue.cc:260] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.684986 18550 raft_consensus.cc:3060] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.689353 18475 raft_consensus.cc:2468] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:37.690846 18367 leader_election.cc:304] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:37.691530 18592 raft_consensus.cc:993] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff: : Instructing follower 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:37.691792 18600 raft_consensus.cc:2804] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.692247 18592 raft_consensus.cc:1081] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Signalling peer 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:37.691540 18550 raft_consensus.cc:2468] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:37.692483 18600 raft_consensus.cc:697] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:37.694371 18475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa0f514d6ce1487ab6000263ade7054c"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:58550
I20260709 13:07:37.694482 18600 consensus_queue.cc:237] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.695152 18475 raft_consensus.cc:493] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.695658 18475 raft_consensus.cc:3060] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.703517 18475 raft_consensus.cc:515] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.703629 18298 catalog_manager.cc:5697] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 1 to 2, leader changed from c15cc53a71664205bbbbc284c1d299ff (127.7.213.67) to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.706019 18475 leader_election.cc:290] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:37.707959 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0f514d6ce1487ab6000263ade7054c" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:37.708688 18401 raft_consensus.cc:3060] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.712963 18628 raft_consensus.cc:993] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa: : Instructing follower 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:37.713570 18631 raft_consensus.cc:1081] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Signalling peer 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:37.715510 18400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:48604
I20260709 13:07:37.716251 18400 raft_consensus.cc:493] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:37.716791 18400 raft_consensus.cc:3060] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.716996 18401 raft_consensus.cc:2468] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:37.714958 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0f514d6ce1487ab6000263ade7054c" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:37.718375 18439 leader_election.cc:304] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:37.718690 18550 raft_consensus.cc:3055] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.718957 18550 raft_consensus.cc:740] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:37.719584 18550 consensus_queue.cc:260] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.720589 18550 raft_consensus.cc:3060] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.722944 18400 raft_consensus.cc:515] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.724704 18400 leader_election.cc:290] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:37.725484 18628 raft_consensus.cc:2804] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.725723 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:37.726054 18550 raft_consensus.cc:2468] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:37.726409 18549 raft_consensus.cc:3060] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.726830 18628 raft_consensus.cc:697] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:37.726730 18475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:37.727555 18475 raft_consensus.cc:3055] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:37.727948 18475 raft_consensus.cc:740] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:37.728435 18628 consensus_queue.cc:237] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.728735 18475 consensus_queue.cc:260] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.730087 18475 raft_consensus.cc:3060] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.732980 18549 raft_consensus.cc:2468] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:37.734099 18365 leader_election.cc:304] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:37.734942 18600 raft_consensus.cc:2804] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:37.735450 18600 raft_consensus.cc:697] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:37.736310 18600 consensus_queue.cc:237] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:37.736784 18296 catalog_manager.cc:5697] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 1 to 2, leader changed from c15cc53a71664205bbbbc284c1d299ff (127.7.213.67) to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.738654 18475 raft_consensus.cc:2468] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:37.741742 18475 raft_consensus.cc:3060] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.747483 18298 catalog_manager.cc:5697] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 1 to 2, leader changed from 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66) to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.748147 18475 raft_consensus.cc:1275] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:37.749594 18600 consensus_queue.cc:1048] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:37.762393 18549 raft_consensus.cc:1275] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:37.764290 18611 consensus_queue.cc:1048] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:37.856442 18475 raft_consensus.cc:1275] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:37.857990 18592 consensus_queue.cc:1048] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:37.866732 18400 raft_consensus.cc:1275] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:37.868614 18592 consensus_queue.cc:1048] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:37.871246 18475 raft_consensus.cc:1275] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:37.873024 18600 consensus_queue.cc:1048] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:37.880576 18400 raft_consensus.cc:1275] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:37.883951 18549 raft_consensus.cc:1275] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:37.884660 18628 consensus_queue.cc:1048] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:37.885782 18600 consensus_queue.cc:1048] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:37.896416 18549 raft_consensus.cc:3060] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:37.905146 18549 raft_consensus.cc:1275] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:37.906486 18628 consensus_queue.cc:1048] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.027529 18400 raft_consensus.cc:1275] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.029562 18628 consensus_queue.cc:1048] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.039650 18549 raft_consensus.cc:1275] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.041692 18598 consensus_queue.cc:1048] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.154799 18400 raft_consensus.cc:1275] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.156333 18598 consensus_queue.cc:1048] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.164274 18549 raft_consensus.cc:1275] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.166121 18628 consensus_queue.cc:1048] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.237934 18549 raft_consensus.cc:1275] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.239289 18600 consensus_queue.cc:1048] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.248524 18475 raft_consensus.cc:1275] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.439231 18671 consensus_queue.cc:1048] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.446743 18476 raft_consensus.cc:1275] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.449246 18671 consensus_queue.cc:1048] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:38.465322 18549 raft_consensus.cc:1275] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:38.467142 18671 consensus_queue.cc:1048] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:39.364907 18497 debug-util.cc:398] Leaking SignalData structure 0x7b08000ef120 after lost signal to thread 8024
W20260709 13:07:39.366314 18497 debug-util.cc:398] Leaking SignalData structure 0x7b0800035740 after lost signal to thread 18333
W20260709 13:07:39.367878 18497 debug-util.cc:398] Leaking SignalData structure 0x7b08001b9b40 after lost signal to thread 18425
W20260709 13:07:39.369202 18497 debug-util.cc:398] Leaking SignalData structure 0x7b0800034100 after lost signal to thread 18500
W20260709 13:07:39.370513 18497 debug-util.cc:398] Leaking SignalData structure 0x7b08002f08c0 after lost signal to thread 18574
I20260709 13:07:40.497534 18298 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39464:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:40.501169 18298 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:40.570211 18388 tablet_service.cc:1511] Processing CreateTablet for tablet 00f1ed4b42dc4bb4b840ad717387a021 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:40.573678 18385 tablet_service.cc:1511] Processing CreateTablet for tablet 68e275dbd8004afc8de216691adf256c (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:40.576474 18458 tablet_service.cc:1511] Processing CreateTablet for tablet 00f1ed4b42dc4bb4b840ad717387a021 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:40.577775 18458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f1ed4b42dc4bb4b840ad717387a021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.578377 18533 tablet_service.cc:1511] Processing CreateTablet for tablet 68e275dbd8004afc8de216691adf256c (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:40.575598 18536 tablet_service.cc:1511] Processing CreateTablet for tablet 00f1ed4b42dc4bb4b840ad717387a021 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:40.587793 18388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f1ed4b42dc4bb4b840ad717387a021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.592983 18385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e275dbd8004afc8de216691adf256c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.593815 18383 tablet_service.cc:1511] Processing CreateTablet for tablet 9775f70e55ab4ab39d839e01d281256a (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:40.595412 18383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9775f70e55ab4ab39d839e01d281256a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.597929 18391 tablet_service.cc:1511] Processing CreateTablet for tablet c77c97e9a68a44b5aaef74caedd8440a (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:40.600850 18389 tablet_service.cc:1511] Processing CreateTablet for tablet 522a49a4e17842f9b78405b0a23b9ba9 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:40.602721 18390 tablet_service.cc:1511] Processing CreateTablet for tablet 01c74d4f7b884ba68067c533661fa56a (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:40.602907 18387 tablet_service.cc:1511] Processing CreateTablet for tablet d8e87107984c45c3aa20aed61a040a35 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:40.587859 18533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e275dbd8004afc8de216691adf256c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.590804 18537 tablet_service.cc:1511] Processing CreateTablet for tablet 9775f70e55ab4ab39d839e01d281256a (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:40.609409 18534 tablet_service.cc:1511] Processing CreateTablet for tablet c77c97e9a68a44b5aaef74caedd8440a (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:40.611949 18391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c77c97e9a68a44b5aaef74caedd8440a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.612776 18387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e87107984c45c3aa20aed61a040a35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.619649 18538 tablet_service.cc:1511] Processing CreateTablet for tablet 522a49a4e17842f9b78405b0a23b9ba9 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:40.633252 18461 tablet_service.cc:1511] Processing CreateTablet for tablet 25a4dcc5945a4b1fa3ef1546c6987b14 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:40.634578 18461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a4dcc5945a4b1fa3ef1546c6987b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.620972 18539 tablet_service.cc:1511] Processing CreateTablet for tablet d8e87107984c45c3aa20aed61a040a35 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:40.622629 18535 tablet_service.cc:1511] Processing CreateTablet for tablet 01c74d4f7b884ba68067c533661fa56a (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:40.624679 18540 tablet_service.cc:1511] Processing CreateTablet for tablet 8168a7e324904bdca04258a02582133e (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:40.629065 18532 tablet_service.cc:1511] Processing CreateTablet for tablet 25a4dcc5945a4b1fa3ef1546c6987b14 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:40.629580 18465 tablet_service.cc:1511] Processing CreateTablet for tablet 9775f70e55ab4ab39d839e01d281256a (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:40.641230 18465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9775f70e55ab4ab39d839e01d281256a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.630141 18464 tablet_service.cc:1511] Processing CreateTablet for tablet 68e275dbd8004afc8de216691adf256c (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:40.645408 18464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e275dbd8004afc8de216691adf256c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.631069 18460 tablet_service.cc:1511] Processing CreateTablet for tablet c77c97e9a68a44b5aaef74caedd8440a (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:40.649323 18460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c77c97e9a68a44b5aaef74caedd8440a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.631253 18466 tablet_service.cc:1511] Processing CreateTablet for tablet 522a49a4e17842f9b78405b0a23b9ba9 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:40.653352 18466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522a49a4e17842f9b78405b0a23b9ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.631709 18390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c74d4f7b884ba68067c533661fa56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.656925 18389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522a49a4e17842f9b78405b0a23b9ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.632144 18459 tablet_service.cc:1511] Processing CreateTablet for tablet 01c74d4f7b884ba68067c533661fa56a (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:40.658707 18459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c74d4f7b884ba68067c533661fa56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.632259 18463 tablet_service.cc:1511] Processing CreateTablet for tablet 8168a7e324904bdca04258a02582133e (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
W20260709 13:07:40.660929 18347 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:40.661610 18463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8168a7e324904bdca04258a02582133e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.631806 18534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c77c97e9a68a44b5aaef74caedd8440a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.632319 18462 tablet_service.cc:1511] Processing CreateTablet for tablet d8e87107984c45c3aa20aed61a040a35 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:40.665253 18462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e87107984c45c3aa20aed61a040a35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.668030 18537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9775f70e55ab4ab39d839e01d281256a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.668710 18540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8168a7e324904bdca04258a02582133e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.671355 18532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a4dcc5945a4b1fa3ef1546c6987b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.676098 18535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01c74d4f7b884ba68067c533661fa56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.680073 18538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522a49a4e17842f9b78405b0a23b9ba9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.684172 18539 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e87107984c45c3aa20aed61a040a35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.688150 18536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00f1ed4b42dc4bb4b840ad717387a021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.737051 18386 tablet_service.cc:1511] Processing CreateTablet for tablet 8168a7e324904bdca04258a02582133e (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:40.744778 18386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8168a7e324904bdca04258a02582133e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.748456 18694 tablet_bootstrap.cc:492] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:40.781526 18385 tablet_service.cc:1511] Processing CreateTablet for tablet 25a4dcc5945a4b1fa3ef1546c6987b14 (DEFAULT_TABLE table=soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:40.783094 18385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a4dcc5945a4b1fa3ef1546c6987b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:40.783691 18694 tablet_bootstrap.cc:654] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.816795 18695 tablet_bootstrap.cc:492] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:40.824523 18693 tablet_bootstrap.cc:492] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:40.833879 18695 tablet_bootstrap.cc:654] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.884717 18693 tablet_bootstrap.cc:654] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.900699 18694 tablet_bootstrap.cc:492] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:40.902123 18694 ts_tablet_manager.cc:1403] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.154s	user 0.014s	sys 0.009s
I20260709 13:07:40.964956 18695 tablet_bootstrap.cc:492] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:40.966622 18695 ts_tablet_manager.cc:1403] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.150s	user 0.017s	sys 0.034s
I20260709 13:07:40.947157 18694 raft_consensus.cc:359] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:40.968729 18694 raft_consensus.cc:385] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.969178 18694 raft_consensus.cc:740] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.970212 18694 consensus_queue.cc:260] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:40.977239 18695 raft_consensus.cc:359] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
W20260709 13:07:40.984835 18365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:35945, user_credentials={real_user=slave}} blocked reactor thread for 93641.9us
I20260709 13:07:40.988725 18695 raft_consensus.cc:385] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.998454 18694 ts_tablet_manager.cc:1434] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.096s	user 0.006s	sys 0.005s
W20260709 13:07:40.999855 18367 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.66:45183, user_credentials={real_user=slave}} blocked reactor thread for 47847.8us
I20260709 13:07:41.009182 18694 tablet_bootstrap.cc:492] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:41.012534 18698 raft_consensus.cc:493] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.013252 18698 raft_consensus.cc:515] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:40.993927 18695 raft_consensus.cc:740] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.022099 18693 tablet_bootstrap.cc:492] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:41.021922 18695 consensus_queue.cc:260] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.022895 18693 ts_tablet_manager.cc:1403] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.199s	user 0.016s	sys 0.035s
I20260709 13:07:41.024322 18695 ts_tablet_manager.cc:1434] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.057s	user 0.014s	sys 0.001s
I20260709 13:07:41.024389 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:41.026327 18695 tablet_bootstrap.cc:492] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:41.019706 18698 leader_election.cc:290] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.027357 18693 raft_consensus.cc:359] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.036453 18693 raft_consensus.cc:385] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.036844 18693 raft_consensus.cc:740] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.039665 18694 tablet_bootstrap.cc:654] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.040480 18693 consensus_queue.cc:260] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
W20260709 13:07:41.040520 18365 leader_election.cc:343] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.045691 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:41.043584 18693 ts_tablet_manager.cc:1434] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.006s
I20260709 13:07:41.047904 18693 tablet_bootstrap.cc:492] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:41.055100 18693 tablet_bootstrap.cc:654] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:41.064155 18367 leader_election.cc:343] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.064953 18367 leader_election.cc:304] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5; no voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:41.066227 18698 raft_consensus.cc:2749] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.072878 18695 tablet_bootstrap.cc:654] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.080080 18694 tablet_bootstrap.cc:492] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:41.086320 18694 ts_tablet_manager.cc:1403] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.078s	user 0.016s	sys 0.012s
I20260709 13:07:41.093194 18693 tablet_bootstrap.cc:492] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:41.090744 18694 raft_consensus.cc:359] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.093813 18694 raft_consensus.cc:385] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.093905 18693 ts_tablet_manager.cc:1403] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.004s
I20260709 13:07:41.094218 18694 raft_consensus.cc:740] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.094863 18695 tablet_bootstrap.cc:492] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:41.095407 18695 ts_tablet_manager.cc:1403] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.070s	user 0.018s	sys 0.002s
I20260709 13:07:41.095224 18694 consensus_queue.cc:260] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.098340 18676 raft_consensus.cc:493] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.098862 18676 raft_consensus.cc:515] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.099936 18695 raft_consensus.cc:359] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.100719 18695 raft_consensus.cc:385] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.101012 18695 raft_consensus.cc:740] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.101167 18693 raft_consensus.cc:359] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.102257 18693 raft_consensus.cc:385] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.101944 18695 consensus_queue.cc:260] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.102630 18693 raft_consensus.cc:740] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.104758 18695 ts_tablet_manager.cc:1434] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:07:41.105031 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.106082 18695 tablet_bootstrap.cc:492] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:41.103950 18693 consensus_queue.cc:260] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
W20260709 13:07:41.108203 18439 leader_election.cc:343] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.108667 18693 ts_tablet_manager.cc:1434] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.006s
I20260709 13:07:41.109969 18693 tablet_bootstrap.cc:492] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:41.114678 18695 tablet_bootstrap.cc:654] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.119884 18693 tablet_bootstrap.cc:654] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.121673 18676 leader_election.cc:290] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.122212 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
W20260709 13:07:41.124223 18440 leader_election.cc:343] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.124787 18440 leader_election.cc:304] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa; no voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:41.128265 18695 tablet_bootstrap.cc:492] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:41.128775 18695 ts_tablet_manager.cc:1403] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.007s
I20260709 13:07:41.132215 18676 raft_consensus.cc:2749] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.132052 18695 raft_consensus.cc:359] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.132776 18695 raft_consensus.cc:385] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.133064 18695 raft_consensus.cc:740] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.133517 18694 ts_tablet_manager.cc:1434] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 13:07:41.133986 18695 consensus_queue.cc:260] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.134747 18694 tablet_bootstrap.cc:492] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:41.136554 18695 ts_tablet_manager.cc:1434] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:07:41.139004 18695 tablet_bootstrap.cc:492] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:41.145948 18695 tablet_bootstrap.cc:654] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.157054 18694 tablet_bootstrap.cc:654] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.158077 18695 tablet_bootstrap.cc:492] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:41.158669 18695 ts_tablet_manager.cc:1403] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:07:41.158736 18693 tablet_bootstrap.cc:492] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:41.159437 18693 ts_tablet_manager.cc:1403] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.050s	user 0.022s	sys 0.017s
I20260709 13:07:41.163031 18693 raft_consensus.cc:359] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.164018 18693 raft_consensus.cc:385] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.164374 18693 raft_consensus.cc:740] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.165243 18693 consensus_queue.cc:260] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.167943 18693 ts_tablet_manager.cc:1434] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:41.165812 18695 raft_consensus.cc:359] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.170027 18695 raft_consensus.cc:385] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.170446 18695 raft_consensus.cc:740] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.169823 18693 tablet_bootstrap.cc:492] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:41.171663 18695 consensus_queue.cc:260] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.177443 18695 ts_tablet_manager.cc:1434] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:07:41.178803 18695 tablet_bootstrap.cc:492] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:41.180063 18693 tablet_bootstrap.cc:654] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.185753 18694 tablet_bootstrap.cc:492] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:41.186408 18694 ts_tablet_manager.cc:1403] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.011s
I20260709 13:07:41.188542 18695 tablet_bootstrap.cc:654] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.189790 18694 raft_consensus.cc:359] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.190618 18694 raft_consensus.cc:385] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.191111 18694 raft_consensus.cc:740] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.192407 18694 consensus_queue.cc:260] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.195721 18694 ts_tablet_manager.cc:1434] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:07:41.197284 18694 tablet_bootstrap.cc:492] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:41.204895 18694 tablet_bootstrap.cc:654] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.208034 18695 tablet_bootstrap.cc:492] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:41.208709 18695 ts_tablet_manager.cc:1403] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.010s
I20260709 13:07:41.211467 18695 raft_consensus.cc:359] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.212210 18695 raft_consensus.cc:385] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.212754 18695 raft_consensus.cc:740] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.213788 18695 consensus_queue.cc:260] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.216432 18695 ts_tablet_manager.cc:1434] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:41.217888 18695 tablet_bootstrap.cc:492] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:41.218519 18693 tablet_bootstrap.cc:492] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:41.219131 18693 ts_tablet_manager.cc:1403] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.050s	user 0.030s	sys 0.008s
I20260709 13:07:41.222339 18693 raft_consensus.cc:359] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.223109 18693 raft_consensus.cc:385] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.223462 18693 raft_consensus.cc:740] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.224246 18693 consensus_queue.cc:260] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.226756 18693 ts_tablet_manager.cc:1434] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 13:07:41.227262 18695 tablet_bootstrap.cc:654] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.228071 18693 tablet_bootstrap.cc:492] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:41.235826 18693 tablet_bootstrap.cc:654] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.245048 18693 tablet_bootstrap.cc:492] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:41.245048 18695 tablet_bootstrap.cc:492] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:41.245652 18693 ts_tablet_manager.cc:1403] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 13:07:41.245689 18695 ts_tablet_manager.cc:1403] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.017s
I20260709 13:07:41.247066 18694 tablet_bootstrap.cc:492] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:41.247545 18694 ts_tablet_manager.cc:1403] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.051s	user 0.022s	sys 0.001s
I20260709 13:07:41.249003 18693 raft_consensus.cc:359] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.249830 18693 raft_consensus.cc:385] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.250036 18694 raft_consensus.cc:359] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.250144 18693 raft_consensus.cc:740] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.250483 18695 raft_consensus.cc:359] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.250905 18694 raft_consensus.cc:385] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.251240 18695 raft_consensus.cc:385] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.251291 18694 raft_consensus.cc:740] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.251680 18695 raft_consensus.cc:740] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.251484 18693 consensus_queue.cc:260] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.252416 18694 consensus_queue.cc:260] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.254101 18693 ts_tablet_manager.cc:1434] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:41.255205 18693 tablet_bootstrap.cc:492] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:41.262811 18694 ts_tablet_manager.cc:1434] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:07:41.265067 18694 tablet_bootstrap.cc:492] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:41.267673 18693 tablet_bootstrap.cc:654] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.269006 18625 raft_consensus.cc:493] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.269524 18625 raft_consensus.cc:515] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.263227 18695 consensus_queue.cc:260] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.274024 18625 leader_election.cc:290] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.274292 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a4dcc5945a4b1fa3ef1546c6987b14" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:41.275600 18474 raft_consensus.cc:2468] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:41.276250 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a4dcc5945a4b1fa3ef1546c6987b14" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.276479 18676 raft_consensus.cc:493] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.277024 18676 raft_consensus.cc:515] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
W20260709 13:07:41.277833 18513 leader_election.cc:343] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.278811 18695 ts_tablet_manager.cc:1434] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.033s	user 0.004s	sys 0.004s
I20260709 13:07:41.279248 18676 leader_election.cc:290] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.280341 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:41.280727 18516 leader_election.cc:304] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:41.280951 18549 raft_consensus.cc:2468] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:41.282241 18440 leader_election.cc:304] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:41.282657 18695 tablet_bootstrap.cc:492] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:41.283867 18676 raft_consensus.cc:2804] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.284139 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.284466 18676 raft_consensus.cc:493] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.282349 18625 raft_consensus.cc:2804] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:41.285497 18439 leader_election.cc:343] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.285012 18676 raft_consensus.cc:3060] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.285602 18625 raft_consensus.cc:493] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.286580 18625 raft_consensus.cc:3060] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.297277 18693 tablet_bootstrap.cc:492] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:41.297915 18693 ts_tablet_manager.cc:1403] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.043s	user 0.003s	sys 0.008s
I20260709 13:07:41.299314 18695 tablet_bootstrap.cc:654] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.307060 18694 tablet_bootstrap.cc:654] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.309418 18625 raft_consensus.cc:515] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.310923 18676 raft_consensus.cc:515] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.311859 18695 tablet_bootstrap.cc:492] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:41.312526 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a4dcc5945a4b1fa3ef1546c6987b14" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:41.306201 18693 raft_consensus.cc:359] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.314009 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:41.314244 18693 raft_consensus.cc:385] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:41.314291 18513 leader_election.cc:343] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.314707 18693 raft_consensus.cc:740] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.315193 18695 ts_tablet_manager.cc:1403] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.010s
W20260709 13:07:41.315430 18439 leader_election.cc:343] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.315498 18693 consensus_queue.cc:260] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.318061 18693 ts_tablet_manager.cc:1434] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.004s
I20260709 13:07:41.319010 18693 tablet_bootstrap.cc:492] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:41.319084 18695 raft_consensus.cc:359] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.320020 18695 raft_consensus.cc:385] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.320109 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:41.320408 18695 raft_consensus.cc:740] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.320693 18549 raft_consensus.cc:3060] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.321499 18695 consensus_queue.cc:260] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.324136 18680 raft_consensus.cc:493] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.324625 18680 raft_consensus.cc:515] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.326747 18680 leader_election.cc:290] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.328310 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522a49a4e17842f9b78405b0a23b9ba9" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:41.328974 18474 raft_consensus.cc:2468] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 0.
I20260709 13:07:41.330199 18717 raft_consensus.cc:493] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.330865 18367 leader_election.cc:304] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
W20260709 13:07:41.332651 18516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.66:45183, user_credentials={real_user=slave}} blocked reactor thread for 52136.1us
I20260709 13:07:41.332753 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522a49a4e17842f9b78405b0a23b9ba9" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:41.333353 18680 raft_consensus.cc:2804] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.333531 18550 raft_consensus.cc:2468] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 0.
I20260709 13:07:41.333896 18680 raft_consensus.cc:493] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.334447 18680 raft_consensus.cc:3060] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.331935 18714 raft_consensus.cc:493] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.339349 18714 raft_consensus.cc:515] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.332314 18695 ts_tablet_manager.cc:1434] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.003s
I20260709 13:07:41.341667 18695 tablet_bootstrap.cc:492] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
I20260709 13:07:41.345332 18549 raft_consensus.cc:2468] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:41.345942 18717 raft_consensus.cc:515] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.347138 18676 leader_election.cc:290] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.351643 18680 raft_consensus.cc:515] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.355424 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
W20260709 13:07:41.359386 18516 leader_election.cc:343] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.360198 18694 tablet_bootstrap.cc:492] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:41.360659 18680 leader_election.cc:290] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.357156 18695 tablet_bootstrap.cc:654] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.354069 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.362149 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a4dcc5945a4b1fa3ef1546c6987b14" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:41.362824 18474 raft_consensus.cc:3060] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.364696 18676 raft_consensus.cc:493] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:41.364986 18513 leader_election.cc:343] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.365233 18676 raft_consensus.cc:515] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.365491 18513 leader_election.cc:304] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:41.354218 18440 leader_election.cc:304] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:41.353408 18717 leader_election.cc:290] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.366956 18717 raft_consensus.cc:2749] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.360740 18625 leader_election.cc:290] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.367339 18676 leader_election.cc:290] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.368276 18692 raft_consensus.cc:2804] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.368289 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522a49a4e17842f9b78405b0a23b9ba9" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:41.368873 18692 raft_consensus.cc:697] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:41.369318 18476 raft_consensus.cc:3060] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.369837 18692 consensus_queue.cc:237] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.371075 18714 leader_election.cc:290] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.374481 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.375012 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.375519 18693 tablet_bootstrap.cc:654] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.384361 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:41.386853 18550 raft_consensus.cc:2468] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:41.390365 18476 raft_consensus.cc:2468] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 1.
I20260709 13:07:41.352875 18718 raft_consensus.cc:493] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.394912 18440 leader_election.cc:304] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:41.397812 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
W20260709 13:07:41.399447 18513 leader_election.cc:343] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.400079 18718 raft_consensus.cc:515] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.398481 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522a49a4e17842f9b78405b0a23b9ba9" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:41.399653 18694 ts_tablet_manager.cc:1403] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.135s	user 0.008s	sys 0.008s
W20260709 13:07:41.399803 18439 leader_election.cc:343] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.400416 18476 raft_consensus.cc:2468] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:41.401016 18549 raft_consensus.cc:3060] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.394408 18367 leader_election.cc:304] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:41.403720 18718 leader_election.cc:290] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.404204 18721 raft_consensus.cc:493] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.408262 18721 raft_consensus.cc:515] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.408377 18298 catalog_manager.cc:5697] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:41.409332 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.410022 18401 raft_consensus.cc:2468] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:41.410878 18721 leader_election.cc:290] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.411314 18680 raft_consensus.cc:2804] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.410945 18694 raft_consensus.cc:359] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.413440 18513 leader_election.cc:304] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:41.414505 18475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:41.420048 18694 raft_consensus.cc:385] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.415162 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.420948 18474 raft_consensus.cc:2468] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:41.421037 18694 raft_consensus.cc:740] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.421546 18401 raft_consensus.cc:2468] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:41.406339 18676 raft_consensus.cc:2804] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.420060 18680 raft_consensus.cc:697] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:41.421948 18694 consensus_queue.cc:260] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.423501 18680 consensus_queue.cc:237] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.423899 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:41.424621 18676 raft_consensus.cc:493] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.425099 18676 raft_consensus.cc:3060] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.416720 18695 tablet_bootstrap.cc:492] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:41.418252 18516 leader_election.cc:304] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:41.427193 18695 ts_tablet_manager.cc:1403] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.086s	user 0.028s	sys 0.014s
I20260709 13:07:41.427405 18718 raft_consensus.cc:2804] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.427857 18718 raft_consensus.cc:493] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.428017 18720 raft_consensus.cc:2804] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.428289 18718 raft_consensus.cc:3060] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.428514 18720 raft_consensus.cc:493] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:41.428550 18516 leader_election.cc:343] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.429008 18720 raft_consensus.cc:3060] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.430089 18516 leader_election.cc:304] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:41.430946 18714 raft_consensus.cc:2804] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.431430 18714 raft_consensus.cc:697] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:41.432242 18714 consensus_queue.cc:237] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.436563 18694 ts_tablet_manager.cc:1434] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 13:07:41.419358 18693 tablet_bootstrap.cc:492] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:41.418206 18549 raft_consensus.cc:2468] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 1.
I20260709 13:07:41.440538 18693 ts_tablet_manager.cc:1403] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.122s	user 0.013s	sys 0.017s
I20260709 13:07:41.440325 18439 leader_election.cc:304] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:41.441985 18720 raft_consensus.cc:515] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.442903 18695 raft_consensus.cc:359] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.443692 18718 raft_consensus.cc:515] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.444873 18695 raft_consensus.cc:385] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.445297 18695 raft_consensus.cc:740] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.446244 18695 consensus_queue.cc:260] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.446919 18724 raft_consensus.cc:2804] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.447760 18724 raft_consensus.cc:493] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.449175 18724 raft_consensus.cc:3060] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.449579 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:41.450086 18695 ts_tablet_manager.cc:1434] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:07:41.450225 18401 raft_consensus.cc:3060] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.451260 18695 tablet_bootstrap.cc:492] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff: Bootstrap starting.
W20260709 13:07:41.448674 18440 leader_election.cc:343] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.452821 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:41.453442 18720 leader_election.cc:290] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.460743 18676 raft_consensus.cc:515] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
W20260709 13:07:41.456761 18516 leader_election.cc:343] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.456766 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:41.462627 18474 raft_consensus.cc:2393] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c15cc53a71664205bbbbc284c1d299ff in current term 1: Already voted for candidate 9695202e5bb14e639ecbba6b25697dfa in this term.
I20260709 13:07:41.448236 18694 tablet_bootstrap.cc:492] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:41.464896 18399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:41.457129 18718 leader_election.cc:290] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.467823 18695 tablet_bootstrap.cc:654] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.468713 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
W20260709 13:07:41.469213 18439 leader_election.cc:343] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:41.470125 18513 leader_election.cc:343] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.471482 18513 leader_election.cc:304] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:41.472316 18718 raft_consensus.cc:2749] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.468747 18299 catalog_manager.cc:5697] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:41.475101 18676 leader_election.cc:290] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.476025 18401 raft_consensus.cc:2468] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:41.477473 18513 leader_election.cc:304] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:41.477859 18720 raft_consensus.cc:493] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.477465 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:41.478513 18720 raft_consensus.cc:515] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.479405 18549 raft_consensus.cc:2393] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9695202e5bb14e639ecbba6b25697dfa in current term 1: Already voted for candidate c15cc53a71664205bbbbc284c1d299ff in this term.
I20260709 13:07:41.479919 18625 raft_consensus.cc:2804] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.480436 18694 tablet_bootstrap.cc:654] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.480599 18440 leader_election.cc:304] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa; no voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:41.480466 18625 raft_consensus.cc:697] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:41.481851 18625 consensus_queue.cc:237] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.483057 18720 leader_election.cc:290] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.483443 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.484275 18401 raft_consensus.cc:2468] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:41.485823 18513 leader_election.cc:304] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:41.484272 18676 raft_consensus.cc:2749] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.489931 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:41.490746 18720 raft_consensus.cc:2804] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.490873 18474 raft_consensus.cc:2468] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:41.491271 18720 raft_consensus.cc:493] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.495919 18720 raft_consensus.cc:3060] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.487854 18693 raft_consensus.cc:359] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.496472 18693 raft_consensus.cc:385] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.483860 18296 catalog_manager.cc:5697] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 0 to 1, leader changed from <none> to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 1 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.494115 18724 raft_consensus.cc:515] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.498831 18693 raft_consensus.cc:740] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.504048 18693 consensus_queue.cc:260] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.509240 18693 ts_tablet_manager.cc:1434] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.065s	user 0.000s	sys 0.008s
I20260709 13:07:41.509547 18694 tablet_bootstrap.cc:492] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:41.510591 18694 ts_tablet_manager.cc:1403] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.019s
I20260709 13:07:41.514940 18694 raft_consensus.cc:359] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.515498 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:41.511466 18693 tablet_bootstrap.cc:492] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
I20260709 13:07:41.516471 18724 leader_election.cc:290] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
W20260709 13:07:41.520617 18440 leader_election.cc:343] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.530819 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:41.531559 18401 raft_consensus.cc:3060] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.538938 18401 raft_consensus.cc:2468] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:41.540055 18693 tablet_bootstrap.cc:654] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.519639 18694 raft_consensus.cc:385] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.544118 18439 leader_election.cc:304] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:41.545687 18676 raft_consensus.cc:2804] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.546415 18676 raft_consensus.cc:697] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:41.541181 18694 raft_consensus.cc:740] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.553066 18694 consensus_queue.cc:260] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.547655 18676 consensus_queue.cc:237] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.559803 18694 ts_tablet_manager.cc:1434] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.044s	user 0.004s	sys 0.001s
I20260709 13:07:41.561580 18694 tablet_bootstrap.cc:492] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:41.567094 18720 raft_consensus.cc:515] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.579913 18695 tablet_bootstrap.cc:492] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff: No bootstrap required, opened a new log
I20260709 13:07:41.579918 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:41.580809 18695 ts_tablet_manager.cc:1403] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff: Time spent bootstrapping tablet: real 0.130s	user 0.022s	sys 0.032s
I20260709 13:07:41.580878 18400 raft_consensus.cc:3060] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.584199 18695 raft_consensus.cc:359] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.584764 18695 raft_consensus.cc:385] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.585071 18695 raft_consensus.cc:740] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.585923 18695 consensus_queue.cc:260] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.589152 18625 raft_consensus.cc:493] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.589545 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:41.589656 18625 raft_consensus.cc:515] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.590350 18474 raft_consensus.cc:3060] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.591714 18400 raft_consensus.cc:2468] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:41.592242 18625 leader_election.cc:290] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.593760 18513 leader_election.cc:304] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:41.594475 18693 tablet_bootstrap.cc:492] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:41.595120 18693 ts_tablet_manager.cc:1403] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.084s	user 0.020s	sys 0.025s
I20260709 13:07:41.598031 18714 raft_consensus.cc:2804] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.598701 18714 raft_consensus.cc:697] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:41.598801 18718 raft_consensus.cc:493] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.599066 18474 raft_consensus.cc:2468] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
I20260709 13:07:41.599429 18718 raft_consensus.cc:515] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.599795 18714 consensus_queue.cc:237] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.601930 18718 leader_election.cc:290] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.603838 18475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:41.598541 18693 raft_consensus.cc:359] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.605921 18693 raft_consensus.cc:385] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.606269 18693 raft_consensus.cc:740] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.607178 18693 consensus_queue.cc:260] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.608258 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8168a7e324904bdca04258a02582133e" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:41.610033 18693 ts_tablet_manager.cc:1434] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:07:41.611380 18693 tablet_bootstrap.cc:492] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa: Bootstrap starting.
W20260709 13:07:41.612628 18516 leader_election.cc:343] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.613457 18720 leader_election.cc:290] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.616236 18694 tablet_bootstrap.cc:654] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.617998 18693 tablet_bootstrap.cc:654] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.618367 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8168a7e324904bdca04258a02582133e" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.619165 18400 raft_consensus.cc:2468] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 0.
I20260709 13:07:41.617252 18695 ts_tablet_manager.cc:1434] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 13:07:41.621009 18513 leader_election.cc:304] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:41.622772 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.623693 18400 raft_consensus.cc:2393] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c15cc53a71664205bbbbc284c1d299ff in current term 1: Already voted for candidate 9695202e5bb14e639ecbba6b25697dfa in this term.
I20260709 13:07:41.624598 18625 raft_consensus.cc:2804] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.625053 18625 raft_consensus.cc:493] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.626514 18625 raft_consensus.cc:3060] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.628867 18694 tablet_bootstrap.cc:492] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:41.629190 18513 leader_election.cc:304] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c15cc53a71664205bbbbc284c1d299ff; no voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:41.629532 18694 ts_tablet_manager.cc:1403] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.000s
I20260709 13:07:41.634123 18720 raft_consensus.cc:3060] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.645426 18625 raft_consensus.cc:515] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.648857 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8168a7e324904bdca04258a02582133e" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:41.649712 18400 raft_consensus.cc:3060] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.649921 18297 catalog_manager.cc:5697] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:41.635260 18694 raft_consensus.cc:359] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.656437 18694 raft_consensus.cc:385] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.657014 18694 raft_consensus.cc:740] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.657792 18694 consensus_queue.cc:260] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.660189 18694 ts_tablet_manager.cc:1434] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:07:41.661237 18694 tablet_bootstrap.cc:492] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:41.656328 18720 raft_consensus.cc:2749] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:41.669435 18347 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:41.672370 18625 leader_election.cc:290] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:41.675439 18693 tablet_bootstrap.cc:492] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa: No bootstrap required, opened a new log
I20260709 13:07:41.675806 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8168a7e324904bdca04258a02582133e" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:41.677253 18400 raft_consensus.cc:2468] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 1.
W20260709 13:07:41.677486 18516 leader_election.cc:343] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.676025 18693 ts_tablet_manager.cc:1403] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa: Time spent bootstrapping tablet: real 0.065s	user 0.028s	sys 0.015s
I20260709 13:07:41.678586 18513 leader_election.cc:304] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:41.679855 18625 raft_consensus.cc:2804] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.680387 18625 raft_consensus.cc:697] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:41.680714 18693 raft_consensus.cc:359] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.681509 18693 raft_consensus.cc:385] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.681864 18693 raft_consensus.cc:740] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.681329 18625 consensus_queue.cc:237] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.682538 18693 consensus_queue.cc:260] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.684692 18693 ts_tablet_manager.cc:1434] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:41.685772 18296 catalog_manager.cc:5697] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 0 to 1, leader changed from <none> to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 1 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.689312 18296 catalog_manager.cc:5697] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 0 to 1, leader changed from <none> to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 1 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.691638 18680 raft_consensus.cc:493] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.692143 18694 tablet_bootstrap.cc:654] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.692193 18680 raft_consensus.cc:515] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.697628 18680 leader_election.cc:290] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.698801 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:41.699536 18549 raft_consensus.cc:2393] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45e64265fd5046af90eefab4e50526f5 in current term 1: Already voted for candidate 9695202e5bb14e639ecbba6b25697dfa in this term.
I20260709 13:07:41.701042 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:41.702622 18367 leader_election.cc:304] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5; no voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:41.704265 18680 raft_consensus.cc:3060] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.711158 18680 raft_consensus.cc:2749] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.712216 18694 tablet_bootstrap.cc:492] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:41.712759 18694 ts_tablet_manager.cc:1403] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.052s	user 0.002s	sys 0.011s
I20260709 13:07:41.715587 18694 raft_consensus.cc:359] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.716743 18694 raft_consensus.cc:385] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.717558 18694 raft_consensus.cc:740] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.717048 18296 catalog_manager.cc:5697] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 0 to 1, leader changed from <none> to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 1 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.718338 18694 consensus_queue.cc:260] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.720232 18694 ts_tablet_manager.cc:1434] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:07:41.721266 18694 tablet_bootstrap.cc:492] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5: Bootstrap starting.
I20260709 13:07:41.726689 18694 tablet_bootstrap.cc:654] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.733794 18694 tablet_bootstrap.cc:492] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5: No bootstrap required, opened a new log
I20260709 13:07:41.734473 18694 ts_tablet_manager.cc:1403] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 13:07:41.736510 18694 raft_consensus.cc:359] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.737015 18694 raft_consensus.cc:385] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.737217 18694 raft_consensus.cc:740] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.737823 18694 consensus_queue.cc:260] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.739459 18694 ts_tablet_manager.cc:1434] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.000s
I20260709 13:07:41.809993 18698 consensus_queue.cc:1048] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.830471 18698 consensus_queue.cc:1048] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.875098 18724 raft_consensus.cc:493] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.875619 18724 raft_consensus.cc:515] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.877229 18400 raft_consensus.cc:3060] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.877764 18724 leader_election.cc:290] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.878523 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.879349 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:41.879346 18401 raft_consensus.cc:2393] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9695202e5bb14e639ecbba6b25697dfa in current term 1: Already voted for candidate c15cc53a71664205bbbbc284c1d299ff in this term.
I20260709 13:07:41.882113 18439 leader_election.cc:304] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa; no voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:41.883237 18724 raft_consensus.cc:3060] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.886310 18625 consensus_queue.cc:1048] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:41.889422 18724 raft_consensus.cc:2749] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.898165 18718 consensus_queue.cc:1048] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:41.908898 18724 consensus_queue.cc:1048] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.926833 18718 consensus_queue.cc:1048] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.933808 18676 consensus_queue.cc:1048] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.957720 18717 consensus_queue.cc:1048] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:41.972271 18676 raft_consensus.cc:493] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.972844 18676 raft_consensus.cc:515] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.975144 18676 leader_election.cc:290] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.976317 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:41.976946 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:41.977056 18549 raft_consensus.cc:2468] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:41.977829 18400 raft_consensus.cc:2468] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 0.
I20260709 13:07:41.978693 18440 leader_election.cc:304] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:41.979609 18676 raft_consensus.cc:2804] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.980080 18676 raft_consensus.cc:493] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.980446 18676 raft_consensus.cc:3060] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.980620 18680 raft_consensus.cc:493] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.981278 18680 raft_consensus.cc:515] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.984560 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9695202e5bb14e639ecbba6b25697dfa" is_pre_election: true
I20260709 13:07:41.985494 18474 raft_consensus.cc:2393] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45e64265fd5046af90eefab4e50526f5 in current term 1: Already voted for candidate 9695202e5bb14e639ecbba6b25697dfa in this term.
I20260709 13:07:41.986323 18680 leader_election.cc:290] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.987787 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:41.988970 18549 raft_consensus.cc:2393] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45e64265fd5046af90eefab4e50526f5 in current term 1: Already voted for candidate c15cc53a71664205bbbbc284c1d299ff in this term.
I20260709 13:07:41.989930 18676 raft_consensus.cc:515] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:41.990703 18367 leader_election.cc:304] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5; no voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:41.991622 18680 raft_consensus.cc:3060] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.992895 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:41.993606 18676 leader_election.cc:290] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:41.993577 18400 raft_consensus.cc:3060] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.999276 18680 raft_consensus.cc:2749] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.999547 18400 raft_consensus.cc:2468] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:42.000178 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:42.000860 18549 raft_consensus.cc:3060] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.000797 18439 leader_election.cc:304] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:42.001864 18676 raft_consensus.cc:2804] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:42.002406 18676 raft_consensus.cc:697] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:42.003328 18676 consensus_queue.cc:237] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:42.009454 18549 raft_consensus.cc:2468] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:42.013691 18297 catalog_manager.cc:5697] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 1 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:42.064610 18676 consensus_queue.cc:1048] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.078042 18692 consensus_queue.cc:1048] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.093400 18676 raft_consensus.cc:493] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.093918 18676 raft_consensus.cc:515] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:42.094854 18474 raft_consensus.cc:3060] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.096129 18676 leader_election.cc:290] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:42.096828 18400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8168a7e324904bdca04258a02582133e" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:42.097611 18400 raft_consensus.cc:2393] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9695202e5bb14e639ecbba6b25697dfa in current term 1: Already voted for candidate c15cc53a71664205bbbbc284c1d299ff in this term.
I20260709 13:07:42.097724 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8168a7e324904bdca04258a02582133e" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:42.099793 18439 leader_election.cc:304] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa; no voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:42.103222 18717 consensus_queue.cc:1048] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.105406 18676 raft_consensus.cc:2749] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:42.109828 18717 consensus_queue.cc:1048] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.114598 18720 consensus_queue.cc:1048] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.147658 18714 consensus_queue.cc:1048] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.468262 18676 consensus_queue.cc:1048] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.481814 18692 consensus_queue.cc:1048] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:42.675417 18347 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:43.193810 18676 raft_consensus.cc:493] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.194377 18676 raft_consensus.cc:515] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.196851 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5" is_pre_election: true
I20260709 13:07:43.197466 18401 raft_consensus.cc:2468] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:43.197573 18676 leader_election.cc:290] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:43.198666 18439 leader_election.cc:304] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:43.198763 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff" is_pre_election: true
I20260709 13:07:43.199591 18676 raft_consensus.cc:2804] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:43.200019 18676 raft_consensus.cc:493] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.199681 18550 raft_consensus.cc:2468] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 1.
I20260709 13:07:43.200333 18676 raft_consensus.cc:3060] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.205150 18676 raft_consensus.cc:515] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.206597 18676 leader_election.cc:290] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:43.207561 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:43.207614 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01c74d4f7b884ba68067c533661fa56a" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:43.208278 18550 raft_consensus.cc:3060] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.208369 18401 raft_consensus.cc:3060] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.213562 18550 raft_consensus.cc:2468] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:43.214583 18440 leader_election.cc:304] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:43.215256 18676 raft_consensus.cc:2804] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:43.215378 18401 raft_consensus.cc:2468] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:43.215682 18676 raft_consensus.cc:697] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:43.216503 18676 consensus_queue.cc:237] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.226303 18296 catalog_manager.cc:5697] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 0 to 2, leader changed from <none> to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.400749 18550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
mode: GRACEFUL
new_leader_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:07:43.401547 18550 raft_consensus.cc:606] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Received request to transfer leadership to TS 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:43.407315 18474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68e275dbd8004afc8de216691adf256c"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
mode: GRACEFUL
new_leader_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:58568
I20260709 13:07:43.407872 18474 raft_consensus.cc:606] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Received request to transfer leadership to TS 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:43.413144 18474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
mode: GRACEFUL
new_leader_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:58568
I20260709 13:07:43.413633 18474 raft_consensus.cc:606] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Received request to transfer leadership to TS 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:43.418617 18474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
mode: GRACEFUL
new_leader_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:58568
I20260709 13:07:43.419137 18474 raft_consensus.cc:606] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Received request to transfer leadership to TS 45e64265fd5046af90eefab4e50526f5
I20260709 13:07:43.420555 18692 raft_consensus.cc:993] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa: : Instructing follower 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:43.421078 18676 raft_consensus.cc:1081] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Signalling peer 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:43.422510 18401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:48604
I20260709 13:07:43.423055 18401 raft_consensus.cc:493] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:43.423408 18401 raft_consensus.cc:3060] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.425372 18400 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "522a49a4e17842f9b78405b0a23b9ba9"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
mode: GRACEFUL
new_leader_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:48610
I20260709 13:07:43.425999 18400 raft_consensus.cc:606] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Received request to transfer leadership to TS 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:43.429320 18401 raft_consensus.cc:515] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.431226 18401 leader_election.cc:290] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:43.431874 18550 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
mode: GRACEFUL
new_leader_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:07:43.432382 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:43.432649 18550 raft_consensus.cc:606] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Received request to transfer leadership to TS 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:43.432680 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9775f70e55ab4ab39d839e01d281256a" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:43.433065 18474 raft_consensus.cc:3055] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:43.433522 18474 raft_consensus.cc:740] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:43.433538 18549 raft_consensus.cc:3060] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.434512 18474 consensus_queue.cc:260] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.435623 18474 raft_consensus.cc:3060] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.444584 18549 raft_consensus.cc:2468] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:43.445837 18365 leader_election.cc:304] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:43.444592 18474 raft_consensus.cc:2468] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:43.454219 18549 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8168a7e324904bdca04258a02582133e"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
mode: GRACEFUL
new_leader_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:56948
I20260709 13:07:43.454574 18770 raft_consensus.cc:2804] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:43.454878 18549 raft_consensus.cc:606] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Received request to transfer leadership to TS 9695202e5bb14e639ecbba6b25697dfa
I20260709 13:07:43.455103 18770 raft_consensus.cc:697] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:43.459029 18769 raft_consensus.cc:993] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5: : Instructing follower 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:43.459673 18760 raft_consensus.cc:1081] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Signalling peer 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:43.456044 18770 consensus_queue.cc:237] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.463059 18475 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "522a49a4e17842f9b78405b0a23b9ba9"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:58560
I20260709 13:07:43.463536 18475 raft_consensus.cc:493] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:43.463888 18475 raft_consensus.cc:3060] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.470773 18475 raft_consensus.cc:515] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.473603 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522a49a4e17842f9b78405b0a23b9ba9" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:43.473883 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522a49a4e17842f9b78405b0a23b9ba9" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:43.474228 18401 raft_consensus.cc:3055] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:43.474431 18549 raft_consensus.cc:3060] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.474610 18401 raft_consensus.cc:740] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:43.473665 18296 catalog_manager.cc:5697] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 1 to 2, leader changed from 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66) to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.475464 18401 consensus_queue.cc:260] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.476640 18401 raft_consensus.cc:3060] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.484287 18401 raft_consensus.cc:2468] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:43.485217 18676 raft_consensus.cc:993] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa: : Instructing follower 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:43.485697 18676 raft_consensus.cc:1081] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Signalling peer 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:43.485585 18439 leader_election.cc:304] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:43.486589 18724 raft_consensus.cc:2804] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:43.487111 18724 raft_consensus.cc:697] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:43.487470 18401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68e275dbd8004afc8de216691adf256c"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:48604
I20260709 13:07:43.488039 18724 consensus_queue.cc:237] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.488026 18401 raft_consensus.cc:493] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:43.491918 18401 raft_consensus.cc:3060] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.495532 18549 raft_consensus.cc:2468] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:43.499117 18298 catalog_manager.cc:5697] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 1 to 2, leader changed from 45e64265fd5046af90eefab4e50526f5 (127.7.213.65) to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.501804 18475 leader_election.cc:290] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:43.501574 18401 raft_consensus.cc:515] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.506003 18401 leader_election.cc:290] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:43.506407 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:43.507031 18549 raft_consensus.cc:3060] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.508309 18475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:43.508911 18475 raft_consensus.cc:3055] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:43.509215 18475 raft_consensus.cc:740] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:43.510085 18475 consensus_queue.cc:260] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.511123 18475 raft_consensus.cc:3060] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.514148 18549 raft_consensus.cc:2468] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:43.515825 18365 leader_election.cc:304] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:43.517045 18770 raft_consensus.cc:2804] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:43.517586 18770 raft_consensus.cc:697] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:43.518039 18475 raft_consensus.cc:2468] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:43.518532 18770 consensus_queue.cc:237] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.527467 18298 catalog_manager.cc:5697] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 1 to 2, leader changed from 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66) to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.542999 18724 raft_consensus.cc:993] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa: : Instructing follower 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:43.543468 18692 raft_consensus.cc:1081] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Signalling peer 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:43.545092 18400 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:48604
I20260709 13:07:43.545657 18400 raft_consensus.cc:493] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:43.546038 18400 raft_consensus.cc:3060] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.546073 18717 raft_consensus.cc:993] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff: : Instructing follower 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:43.546622 18763 raft_consensus.cc:1081] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Signalling peer 45e64265fd5046af90eefab4e50526f5 to start an election
I20260709 13:07:43.547996 18401 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
 from {username='slave'} at 127.0.0.1:48596
I20260709 13:07:43.548358 18401 raft_consensus.cc:493] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:43.548560 18401 raft_consensus.cc:3060] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.553690 18400 raft_consensus.cc:515] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.554387 18401 raft_consensus.cc:515] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.557320 18400 leader_election.cc:290] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:43.557684 18475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:43.559805 18475 raft_consensus.cc:3055] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:43.559645 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:43.560258 18475 raft_consensus.cc:740] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:43.560484 18550 raft_consensus.cc:3055] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:43.560832 18550 raft_consensus.cc:740] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:43.561168 18475 consensus_queue.cc:260] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.561615 18550 consensus_queue.cc:260] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.562371 18475 raft_consensus.cc:3060] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.562757 18550 raft_consensus.cc:3060] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.557742 18401 leader_election.cc:290] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Requested vote from peers 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:43.557919 18549 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c77c97e9a68a44b5aaef74caedd8440a" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:43.558523 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00f1ed4b42dc4bb4b840ad717387a021" candidate_uuid: "45e64265fd5046af90eefab4e50526f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:43.566422 18549 raft_consensus.cc:3060] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.566581 18474 raft_consensus.cc:3060] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.569674 18475 raft_consensus.cc:2468] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:43.570998 18367 leader_election.cc:304] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:43.571952 18770 raft_consensus.cc:2804] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:43.572448 18770 raft_consensus.cc:697] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:43.573249 18770 consensus_queue.cc:237] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.576208 18474 raft_consensus.cc:2468] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:43.582017 18550 raft_consensus.cc:2468] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:43.582947 18549 raft_consensus.cc:2468] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45e64265fd5046af90eefab4e50526f5 in term 2.
I20260709 13:07:43.585036 18367 leader_election.cc:304] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:43.586007 18771 raft_consensus.cc:2804] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:43.586655 18771 raft_consensus.cc:697] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Leader. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:43.587414 18771 consensus_queue.cc:237] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.594293 18298 catalog_manager.cc:5697] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 1 to 2, leader changed from 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66) to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.600015 18401 raft_consensus.cc:1275] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:43.601409 18724 consensus_queue.cc:1048] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:43.611723 18298 catalog_manager.cc:5697] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 reported cstate change: term changed from 1 to 2, leader changed from c15cc53a71664205bbbbc284c1d299ff (127.7.213.67) to 45e64265fd5046af90eefab4e50526f5 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "45e64265fd5046af90eefab4e50526f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.617280 18550 raft_consensus.cc:1275] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:43.620476 18724 consensus_queue.cc:1048] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:43.643352 18717 raft_consensus.cc:993] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff: : Instructing follower 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:43.643849 18763 raft_consensus.cc:1081] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Signalling peer 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:43.645368 18474 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8168a7e324904bdca04258a02582133e"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:58550
I20260709 13:07:43.645893 18474 raft_consensus.cc:493] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:43.646240 18474 raft_consensus.cc:3060] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.651435 18474 raft_consensus.cc:515] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.653107 18474 leader_election.cc:290] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:43.653904 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8168a7e324904bdca04258a02582133e" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:43.654313 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8168a7e324904bdca04258a02582133e" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:43.654596 18401 raft_consensus.cc:3060] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.654999 18550 raft_consensus.cc:3055] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:43.655371 18550 raft_consensus.cc:740] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:43.656257 18550 consensus_queue.cc:260] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.657605 18550 raft_consensus.cc:3060] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.661942 18401 raft_consensus.cc:2468] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:43.663165 18439 leader_election.cc:304] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:43.663923 18724 raft_consensus.cc:2804] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:43.664359 18724 raft_consensus.cc:697] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:43.664772 18550 raft_consensus.cc:2468] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:43.665071 18724 consensus_queue.cc:237] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.672610 18296 catalog_manager.cc:5697] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 1 to 2, leader changed from c15cc53a71664205bbbbc284c1d299ff (127.7.213.67) to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.716349 18717 raft_consensus.cc:993] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff: : Instructing follower 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:43.716878 18763 raft_consensus.cc:1081] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Signalling peer 9695202e5bb14e639ecbba6b25697dfa to start an election
I20260709 13:07:43.718628 18474 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
 from {username='slave'} at 127.0.0.1:58550
I20260709 13:07:43.719129 18474 raft_consensus.cc:493] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:43.719429 18474 raft_consensus.cc:3060] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.724859 18474 raft_consensus.cc:515] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.726255 18474 leader_election.cc:290] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), c15cc53a71664205bbbbc284c1d299ff (127.7.213.67:35945)
I20260709 13:07:43.727074 18401 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:43.727454 18550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "9695202e5bb14e639ecbba6b25697dfa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
I20260709 13:07:43.727806 18401 raft_consensus.cc:3060] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.728147 18550 raft_consensus.cc:3055] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:43.728494 18550 raft_consensus.cc:740] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:43.729352 18550 consensus_queue.cc:260] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.730563 18550 raft_consensus.cc:3060] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:43.734720 18401 raft_consensus.cc:2468] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:43.735996 18439 leader_election.cc:304] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, 9695202e5bb14e639ecbba6b25697dfa; no voters: 
I20260709 13:07:43.736792 18724 raft_consensus.cc:2804] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:43.737339 18724 raft_consensus.cc:697] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Becoming Leader. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:43.737370 18550 raft_consensus.cc:2468] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9695202e5bb14e639ecbba6b25697dfa in term 2.
I20260709 13:07:43.738266 18724 consensus_queue.cc:237] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:43.745868 18296 catalog_manager.cc:5697] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa reported cstate change: term changed from 1 to 2, leader changed from c15cc53a71664205bbbbc284c1d299ff (127.7.213.67) to 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66). New cstate: current_term: 2 leader_uuid: "9695202e5bb14e639ecbba6b25697dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.911062 18401 raft_consensus.cc:1275] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:43.912248 18724 consensus_queue.cc:1048] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:43.920639 18550 raft_consensus.cc:1275] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:43.922246 18676 consensus_queue.cc:1048] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:43.958799 18474 raft_consensus.cc:1275] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:43.960062 18771 consensus_queue.cc:1048] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:43.973145 18550 raft_consensus.cc:1275] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:43.974512 18771 consensus_queue.cc:1048] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:43.999069 18475 raft_consensus.cc:1275] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.000434 18771 consensus_queue.cc:1048] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:44.007272 18549 raft_consensus.cc:1275] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.010479 18770 consensus_queue.cc:1048] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:44.010682 18549 raft_consensus.cc:1275] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.012627 18770 consensus_queue.cc:1048] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:44.023849 18475 raft_consensus.cc:1275] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.025497 18769 consensus_queue.cc:1048] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:44.039975 18549 raft_consensus.cc:1275] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.041374 18789 consensus_queue.cc:1048] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:44.054047 18475 raft_consensus.cc:1275] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Refusing update from remote peer 45e64265fd5046af90eefab4e50526f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.055361 18769 consensus_queue.cc:1048] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:44.097452 18401 raft_consensus.cc:1275] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.098984 18692 consensus_queue.cc:1048] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:44.108229 18550 raft_consensus.cc:1275] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.109942 18676 consensus_queue.cc:1048] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:44.165380 18401 raft_consensus.cc:1275] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.166710 18676 consensus_queue.cc:1048] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:44.175261 18550 raft_consensus.cc:1275] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Refusing update from remote peer 9695202e5bb14e639ecbba6b25697dfa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:44.177031 18692 consensus_queue.cc:1048] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:46.304847 18497 debug-util.cc:398] Leaking SignalData structure 0x7b0800077300 after lost signal to thread 8024
W20260709 13:07:46.305814 18497 debug-util.cc:398] Leaking SignalData structure 0x7b08001cc180 after lost signal to thread 18333
W20260709 13:07:46.307034 18497 debug-util.cc:398] Leaking SignalData structure 0x7b08001ff3e0 after lost signal to thread 18425
W20260709 13:07:46.308054 18497 debug-util.cc:398] Leaking SignalData structure 0x7b0800317d60 after lost signal to thread 18500
W20260709 13:07:46.308914 18497 debug-util.cc:398] Leaking SignalData structure 0x7b08000ab280 after lost signal to thread 18574
I20260709 13:07:46.565984 18298 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39464:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:07:46.587493 18298 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39464:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:07:46.619959  8021 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:07:46.719018 18439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:46203, user_credentials={real_user=slave}} blocked reactor thread for 62814us
I20260709 13:07:46.793459 18843 tablet.cc:1724] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:46.801259 18401 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68e275dbd8004afc8de216691adf256c"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
mode: GRACEFUL
new_leader_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:34566
I20260709 13:07:46.803203 18401 raft_consensus.cc:606] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Received request to transfer leadership to TS c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:46.965462 18854 tablet.cc:1724] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:46.955773 18576 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:25a4dcc5945a4b1fa3ef1546c6987b14)
W20260709 13:07:46.929757 18836 log.cc:927] Time spent T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:07:46.978583 18298 catalog_manager.cc:5789] TS 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203) does not have the latest schema for tablet c77c97e9a68a44b5aaef74caedd8440a (table soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]). Expected version 1 got 0
I20260709 13:07:46.988332 18298 catalog_manager.cc:5789] TS 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203) does not have the latest schema for tablet 00f1ed4b42dc4bb4b840ad717387a021 (table soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]). Expected version 1 got 0
I20260709 13:07:46.991850 18298 catalog_manager.cc:5789] TS 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203) does not have the latest schema for tablet 9775f70e55ab4ab39d839e01d281256a (table soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]). Expected version 1 got 0
I20260709 13:07:46.993644 18298 catalog_manager.cc:5789] TS 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203) does not have the latest schema for tablet 68e275dbd8004afc8de216691adf256c (table soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]). Expected version 1 got 0
I20260709 13:07:46.952891 18853 tablet.cc:1724] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.004703 18855 tablet.cc:1724] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.004657 18852 tablet.cc:1724] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:47.031462 18846 log.cc:927] Time spent T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff: Append to log took a long time: real 0.122s	user 0.003s	sys 0.000s
I20260709 13:07:47.037540 18843 tablet.cc:1724] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.038431 18863 tablet.cc:1724] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.039173 18858 tablet.cc:1724] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.094728 18868 tablet.cc:1724] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:47.100397 18440 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:35945, user_credentials={real_user=slave}} blocked reactor thread for 65912.4us
W20260709 13:07:47.110303 18365 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:35945, user_credentials={real_user=slave}} blocked reactor thread for 65132.3us
I20260709 13:07:47.132925 18870 tablet.cc:1724] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.154280 18865 tablet.cc:1724] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:07:47.173528 18576 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:00f1ed4b42dc4bb4b840ad717387a021)
W20260709 13:07:47.167058 18502 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:01c74d4f7b884ba68067c533661fa56a)
I20260709 13:07:47.197038 18843 tablet.cc:1724] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.229121 18866 raft_consensus.cc:993] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5: : Instructing follower c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:47.229887 18866 raft_consensus.cc:1081] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Signalling peer c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:47.231326 18854 tablet.cc:1724] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.240151 18855 tablet.cc:1724] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.242570 18868 tablet.cc:1724] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.248208 18542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68e275dbd8004afc8de216691adf256c"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:56928
I20260709 13:07:47.249585 18542 raft_consensus.cc:493] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:47.249119 18858 tablet.cc:1724] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.250234 18542 raft_consensus.cc:3060] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:07:47.251382 18439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.65:46203, user_credentials={real_user=slave}} blocked reactor thread for 127769us
I20260709 13:07:47.248076 18865 tablet.cc:1724] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.223459 18852 tablet.cc:1724] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.263579 18474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
mode: GRACEFUL
new_leader_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:48304
I20260709 13:07:47.264276 18474 raft_consensus.cc:606] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Received request to transfer leadership to TS c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:47.271134 18298 catalog_manager.cc:5789] TS 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183) does not have the latest schema for tablet d8e87107984c45c3aa20aed61a040a35 (table soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]). Expected version 1 got 0
I20260709 13:07:47.272580 18298 catalog_manager.cc:5789] TS 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183) does not have the latest schema for tablet 8168a7e324904bdca04258a02582133e (table soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1]). Expected version 1 got 0
I20260709 13:07:47.296779 18878 tablet.cc:1724] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.303279  8021 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:07:47.315304 18853 tablet.cc:1724] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.325637 18863 tablet.cc:1724] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.331827  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:47.342918 18399 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e"
dest_uuid: "45e64265fd5046af90eefab4e50526f5"
mode: GRACEFUL
new_leader_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:34566
I20260709 13:07:47.343598 18399 raft_consensus.cc:606] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Received request to transfer leadership to TS c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:47.350349 18876 raft_consensus.cc:993] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa: : Instructing follower c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:47.351011 18872 raft_consensus.cc:1081] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Signalling peer c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:47.352232 18853 tablet.cc:1724] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.352910 18541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:56938
I20260709 13:07:47.354343 18541 raft_consensus.cc:493] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:47.354743 18541 raft_consensus.cc:3060] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.319924 18854 tablet.cc:1724] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.356681 18474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b"
dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
mode: GRACEFUL
new_leader_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:48304
I20260709 13:07:47.331193 18877 tablet.cc:1724] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.357316 18474 raft_consensus.cc:606] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Received request to transfer leadership to TS c15cc53a71664205bbbbc284c1d299ff
I20260709 13:07:47.358834  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:47.359477  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:47.360608 18870 tablet.cc:1724] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.369150 18855 tablet.cc:1724] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.376168 18843 tablet.cc:1724] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:07:47.381690 18541 raft_consensus.cc:515] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.386859 18399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:47.387509 18399 raft_consensus.cc:3060] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.387989 18474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e87107984c45c3aa20aed61a040a35" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:47.388626 18474 raft_consensus.cc:3055] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:47.388895 18474 raft_consensus.cc:740] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:47.389614 18474 consensus_queue.cc:260] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.390661 18474 raft_consensus.cc:3060] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.386950 18541 leader_election.cc:290] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 3 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:47.399715 18399 raft_consensus.cc:2468] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 3.
I20260709 13:07:47.400655 18542 raft_consensus.cc:515] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.401888 18513 leader_election.cc:304] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:47.401985 18474 raft_consensus.cc:2468] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 3.
I20260709 13:07:47.403106 18838 raft_consensus.cc:2804] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:47.408515 18399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:47.408963 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e275dbd8004afc8de216691adf256c" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:47.409701 18838 raft_consensus.cc:697] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 3 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:47.410837 18399 raft_consensus.cc:3055] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:47.411573 18399 raft_consensus.cc:740] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:47.407949 18542 leader_election.cc:290] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 3 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:47.411863 18838 consensus_queue.cc:237] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.410434 18476 raft_consensus.cc:3060] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.413532 18399 consensus_queue.cc:260] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.418018 18399 raft_consensus.cc:3060] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.418255 18864 raft_consensus.cc:993] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5: : Instructing follower c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:47.418771 18864 raft_consensus.cc:1081] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Signalling peer c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:47.420929 18542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:56928
I20260709 13:07:47.421535 18542 raft_consensus.cc:493] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:47.421933 18542 raft_consensus.cc:3060] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.431248 18399 raft_consensus.cc:2468] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 3.
I20260709 13:07:47.432525 18513 leader_election.cc:304] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:47.433386 18763 raft_consensus.cc:2804] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:47.433847 18763 raft_consensus.cc:697] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 3 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:47.434638 18763 consensus_queue.cc:237] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.439914 18476 raft_consensus.cc:2468] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 3.
I20260709 13:07:47.442504 18298 catalog_manager.cc:6242] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b: soft_deleted_table [id=0ca6ba96be0f4ae19b78ddf5f7ac13c1] alter complete (version 1)
I20260709 13:07:47.443131 18542 raft_consensus.cc:515] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.445787 18542 leader_election.cc:290] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 3 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:47.445919 18399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:47.446552 18399 raft_consensus.cc:3055] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:47.447357 18399 raft_consensus.cc:740] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 45e64265fd5046af90eefab4e50526f5, State: Running, Role: LEADER
I20260709 13:07:47.447785 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6f9c7fa6f8b4a93affd76594fe4ed6e" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:47.448488 18476 raft_consensus.cc:3060] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.448288 18399 consensus_queue.cc:260] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.450486 18399 raft_consensus.cc:3060] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.455349 18298 catalog_manager.cc:5697] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 2 to 3, leader changed from 45e64265fd5046af90eefab4e50526f5 (127.7.213.65) to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 3 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:47.456566 18476 raft_consensus.cc:2468] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 3.
I20260709 13:07:47.457839 18399 raft_consensus.cc:2468] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 3.
I20260709 13:07:47.457760 18516 leader_election.cc:304] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9695202e5bb14e639ecbba6b25697dfa, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:47.457276 18298 catalog_manager.cc:5697] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 2 to 3, leader changed from 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66) to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 3 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:47.458842 18763 raft_consensus.cc:2804] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:47.459306 18763 raft_consensus.cc:697] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 3 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:47.460043 18763 consensus_queue.cc:237] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.472580 18298 catalog_manager.cc:5697] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 2 to 3, leader changed from 45e64265fd5046af90eefab4e50526f5 (127.7.213.65) to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 3 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:47.518193 18876 raft_consensus.cc:993] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa: : Instructing follower c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:47.518694 18872 raft_consensus.cc:1081] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Signalling peer c15cc53a71664205bbbbc284c1d299ff to start an election
I20260709 13:07:47.520116 18542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b"
dest_uuid: "c15cc53a71664205bbbbc284c1d299ff"
 from {username='slave'} at 127.0.0.1:56938
I20260709 13:07:47.520598 18542 raft_consensus.cc:493] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:47.520901 18542 raft_consensus.cc:3060] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.526052 18542 raft_consensus.cc:515] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.527848 18542 leader_election.cc:290] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 3 election: Requested vote from peers 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203), 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183)
I20260709 13:07:47.529127 18476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9695202e5bb14e639ecbba6b25697dfa"
I20260709 13:07:47.529634 18399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2328b3b8dd64a41b451825e2156884b" candidate_uuid: "c15cc53a71664205bbbbc284c1d299ff" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "45e64265fd5046af90eefab4e50526f5"
I20260709 13:07:47.529839 18476 raft_consensus.cc:3055] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:47.530253 18476 raft_consensus.cc:740] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9695202e5bb14e639ecbba6b25697dfa, State: Running, Role: LEADER
I20260709 13:07:47.530323 18399 raft_consensus.cc:3060] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.531143 18476 consensus_queue.cc:260] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.532483 18476 raft_consensus.cc:3060] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:47.537922 18399 raft_consensus.cc:2468] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 3.
I20260709 13:07:47.539211 18513 leader_election.cc:304] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e64265fd5046af90eefab4e50526f5, c15cc53a71664205bbbbc284c1d299ff; no voters: 
I20260709 13:07:47.539572 18476 raft_consensus.cc:2468] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c15cc53a71664205bbbbc284c1d299ff in term 3.
I20260709 13:07:47.540109 18763 raft_consensus.cc:2804] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:47.540643 18763 raft_consensus.cc:697] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 3 LEADER]: Becoming Leader. State: Replica: c15cc53a71664205bbbbc284c1d299ff, State: Running, Role: LEADER
I20260709 13:07:47.541546 18763 consensus_queue.cc:237] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } }
I20260709 13:07:47.549094 18298 catalog_manager.cc:5697] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff reported cstate change: term changed from 2 to 3, leader changed from 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66) to c15cc53a71664205bbbbc284c1d299ff (127.7.213.67). New cstate: current_term: 3 leader_uuid: "c15cc53a71664205bbbbc284c1d299ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c15cc53a71664205bbbbc284c1d299ff" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 35945 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:47.867473 18399 raft_consensus.cc:1275] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:47.868752 18763 consensus_queue.cc:1048] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:47.879514 18476 raft_consensus.cc:1275] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:47.881348 18838 consensus_queue.cc:1048] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:47.891464 18399 raft_consensus.cc:1275] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:47.892905 18763 consensus_queue.cc:1048] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:47.902846 18474 raft_consensus.cc:1275] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:47.906854 18887 consensus_queue.cc:1048] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:47.970963 18399 raft_consensus.cc:1275] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:47.972402 18887 consensus_queue.cc:1048] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:47.984881 18476 raft_consensus.cc:1275] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:07:47.986709 18892 consensus_queue.cc:1048] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:47.994704 18399 raft_consensus.cc:1275] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:47.996084 18892 consensus_queue.cc:1048] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e64265fd5046af90eefab4e50526f5" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.010128 18476 raft_consensus.cc:1275] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Refusing update from remote peer c15cc53a71664205bbbbc284c1d299ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:48.011504 18887 consensus_queue.cc:1048] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9695202e5bb14e639ecbba6b25697dfa" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:48.423259  8021 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:07:48.428267  8021 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:07:48.428679  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:48.433393  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:07:48.433794  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:49.462702  8021 tablet_server.cc:179] TabletServer@127.7.213.65:0 shutting down...
W20260709 13:07:49.467268 18439 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 13:07:49.475636 18439 consensus_peers.cc:597] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa -> Peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Couldn't send request to peer 45e64265fd5046af90eefab4e50526f5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:49.486341 18439 consensus_peers.cc:597] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa -> Peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Couldn't send request to peer 45e64265fd5046af90eefab4e50526f5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:49.496496  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:49.497398  8021 tablet_replica.cc:333] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.498000  8021 raft_consensus.cc:2243] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.498786  8021 raft_consensus.cc:2272] T c77c97e9a68a44b5aaef74caedd8440a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.500737  8021 tablet_replica.cc:333] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.501289  8021 raft_consensus.cc:2243] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.502003  8021 raft_consensus.cc:2272] T 9775f70e55ab4ab39d839e01d281256a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.504051  8021 tablet_replica.cc:333] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.504599  8021 raft_consensus.cc:2243] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.505014  8021 raft_consensus.cc:2272] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.506675  8021 tablet_replica.cc:333] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.507205  8021 raft_consensus.cc:2243] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.507970  8021 raft_consensus.cc:2272] T 00f1ed4b42dc4bb4b840ad717387a021 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.509788  8021 tablet_replica.cc:333] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.510319  8021 raft_consensus.cc:2243] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.510710  8021 raft_consensus.cc:2272] T d3c63b593e614ba995254f08593ea75b P 45e64265fd5046af90eefab4e50526f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.512732  8021 tablet_replica.cc:333] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.513239  8021 raft_consensus.cc:2243] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.513639  8021 raft_consensus.cc:2272] T d8e87107984c45c3aa20aed61a040a35 P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.515234  8021 tablet_replica.cc:333] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.515796  8021 raft_consensus.cc:2243] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.516218  8021 raft_consensus.cc:2272] T 68e275dbd8004afc8de216691adf256c P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.518110  8021 tablet_replica.cc:333] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.518703  8021 raft_consensus.cc:2243] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.519120  8021 raft_consensus.cc:2272] T 6885c0e03a4a403d9cdaad7803be4963 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.521098  8021 tablet_replica.cc:333] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.521617  8021 raft_consensus.cc:2243] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.522034  8021 raft_consensus.cc:2272] T 8168a7e324904bdca04258a02582133e P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.524040  8021 tablet_replica.cc:333] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.524636  8021 raft_consensus.cc:2243] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.525082  8021 raft_consensus.cc:2272] T b465daf7d17e4407b2226ba37042529c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.527704  8021 tablet_replica.cc:333] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
W20260709 13:07:49.528395 18513 consensus_peers.cc:597] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff -> Peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Couldn't send request to peer 45e64265fd5046af90eefab4e50526f5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:49.528707  8021 raft_consensus.cc:2243] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.529477  8021 raft_consensus.cc:2272] T fa0f514d6ce1487ab6000263ade7054c P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.532214  8021 tablet_replica.cc:333] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.532979  8021 raft_consensus.cc:2243] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.534044  8021 raft_consensus.cc:2272] T 930abb26f64e42619494dba489c18df1 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.536804  8021 tablet_replica.cc:333] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.537472  8021 raft_consensus.cc:2243] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.537863  8021 raft_consensus.cc:2272] T 522a49a4e17842f9b78405b0a23b9ba9 P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.539480  8021 tablet_replica.cc:333] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.540017  8021 raft_consensus.cc:2243] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.540752  8021 raft_consensus.cc:2272] T 0bda43ca086543e7b9665d96bdd452fa P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.542953  8021 tablet_replica.cc:333] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.543514  8021 raft_consensus.cc:2243] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.544442  8021 raft_consensus.cc:2272] T ac609b21527a4972b105bdb06f4caaca P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.546869  8021 tablet_replica.cc:333] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.547446  8021 raft_consensus.cc:2243] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.547915  8021 raft_consensus.cc:2272] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.549839  8021 tablet_replica.cc:333] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.550328  8021 raft_consensus.cc:2243] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.550733  8021 raft_consensus.cc:2272] T 01c74d4f7b884ba68067c533661fa56a P 45e64265fd5046af90eefab4e50526f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.552459  8021 tablet_replica.cc:333] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5: stopping tablet replica
I20260709 13:07:49.552951  8021 raft_consensus.cc:2243] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.553386  8021 raft_consensus.cc:2272] T b2328b3b8dd64a41b451825e2156884b P 45e64265fd5046af90eefab4e50526f5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.594784  8021 tablet_server.cc:196] TabletServer@127.7.213.65:0 shutdown complete.
I20260709 13:07:49.626207  8021 tablet_server.cc:179] TabletServer@127.7.213.66:0 shutting down...
I20260709 13:07:49.653896  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:49.654758  8021 tablet_replica.cc:333] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.655381  8021 raft_consensus.cc:2243] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.655932  8021 raft_consensus.cc:2272] T c77c97e9a68a44b5aaef74caedd8440a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.657758  8021 tablet_replica.cc:333] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.658301  8021 raft_consensus.cc:2243] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.658721  8021 raft_consensus.cc:2272] T 68e275dbd8004afc8de216691adf256c P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.660759  8021 tablet_replica.cc:333] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.661291  8021 raft_consensus.cc:2243] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.661785  8021 raft_consensus.cc:2272] T 9775f70e55ab4ab39d839e01d281256a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:49.663184 18516 consensus_peers.cc:597] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff -> Peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Couldn't send request to peer 9695202e5bb14e639ecbba6b25697dfa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:49.663657  8021 tablet_replica.cc:333] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.664355  8021 raft_consensus.cc:2243] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.664784  8021 raft_consensus.cc:2272] T 25a4dcc5945a4b1fa3ef1546c6987b14 P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.666652  8021 tablet_replica.cc:333] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.667232  8021 raft_consensus.cc:2243] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.667658  8021 raft_consensus.cc:2272] T 00f1ed4b42dc4bb4b840ad717387a021 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.669584  8021 tablet_replica.cc:333] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.670220  8021 raft_consensus.cc:2243] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.670661  8021 raft_consensus.cc:2272] T 6885c0e03a4a403d9cdaad7803be4963 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.672925  8021 tablet_replica.cc:333] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.673497  8021 raft_consensus.cc:2243] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.673928  8021 raft_consensus.cc:2272] T d8e87107984c45c3aa20aed61a040a35 P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.675725  8021 tablet_replica.cc:333] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.676323  8021 raft_consensus.cc:2243] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.677093  8021 raft_consensus.cc:2272] T d3c63b593e614ba995254f08593ea75b P 9695202e5bb14e639ecbba6b25697dfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.679023  8021 tablet_replica.cc:333] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.679551  8021 raft_consensus.cc:2243] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.680311  8021 raft_consensus.cc:2272] T 8168a7e324904bdca04258a02582133e P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.682011  8021 tablet_replica.cc:333] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.682547  8021 raft_consensus.cc:2243] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.683319  8021 raft_consensus.cc:2272] T b465daf7d17e4407b2226ba37042529c P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.685307  8021 tablet_replica.cc:333] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.685868  8021 raft_consensus.cc:2243] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.686311  8021 raft_consensus.cc:2272] T 930abb26f64e42619494dba489c18df1 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.688349  8021 tablet_replica.cc:333] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.688812  8021 raft_consensus.cc:2243] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.689479  8021 raft_consensus.cc:2272] T fa0f514d6ce1487ab6000263ade7054c P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.691329  8021 tablet_replica.cc:333] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.691812  8021 raft_consensus.cc:2243] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.692499  8021 raft_consensus.cc:2272] T 522a49a4e17842f9b78405b0a23b9ba9 P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.694221  8021 tablet_replica.cc:333] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.694701  8021 raft_consensus.cc:2243] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.695055  8021 raft_consensus.cc:2272] T 0bda43ca086543e7b9665d96bdd452fa P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.697197  8021 tablet_replica.cc:333] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.697690  8021 raft_consensus.cc:2243] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.698040  8021 raft_consensus.cc:2272] T ac609b21527a4972b105bdb06f4caaca P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.699903  8021 tablet_replica.cc:333] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.700364  8021 raft_consensus.cc:2243] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.700744  8021 raft_consensus.cc:2272] T a6f9c7fa6f8b4a93affd76594fe4ed6e P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.702487  8021 tablet_replica.cc:333] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.703015  8021 raft_consensus.cc:2243] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.703837  8021 raft_consensus.cc:2272] T 01c74d4f7b884ba68067c533661fa56a P 9695202e5bb14e639ecbba6b25697dfa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.705592  8021 tablet_replica.cc:333] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa: stopping tablet replica
I20260709 13:07:49.706040  8021 raft_consensus.cc:2243] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.706436  8021 raft_consensus.cc:2272] T b2328b3b8dd64a41b451825e2156884b P 9695202e5bb14e639ecbba6b25697dfa [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:49.759284 18516 consensus_peers.cc:597] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff -> Peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Couldn't send request to peer 9695202e5bb14e639ecbba6b25697dfa. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:45183: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:49.773030  8021 tablet_server.cc:196] TabletServer@127.7.213.66:0 shutdown complete.
I20260709 13:07:49.805291  8021 tablet_server.cc:179] TabletServer@127.7.213.67:0 shutting down...
W20260709 13:07:49.814831 18513 consensus_peers.cc:597] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff -> Peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Couldn't send request to peer 45e64265fd5046af90eefab4e50526f5. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:46203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:49.837037  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:49.837944  8021 tablet_replica.cc:333] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.838690  8021 raft_consensus.cc:2243] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.839620  8021 raft_consensus.cc:2272] T 68e275dbd8004afc8de216691adf256c P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:49.841306 18513 consensus_peers.cc:597] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff -> Peer 45e64265fd5046af90eefab4e50526f5 (127.7.213.65:46203): Couldn't send request to peer 45e64265fd5046af90eefab4e50526f5. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:46203: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:49.841929  8021 tablet_replica.cc:333] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.842617  8021 raft_consensus.cc:2243] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.843343  8021 raft_consensus.cc:2272] T 25a4dcc5945a4b1fa3ef1546c6987b14 P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.845176  8021 tablet_replica.cc:333] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.845691  8021 raft_consensus.cc:2243] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:07:49.846063 18516 consensus_peers.cc:597] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff -> Peer 9695202e5bb14e639ecbba6b25697dfa (127.7.213.66:45183): Couldn't send request to peer 9695202e5bb14e639ecbba6b25697dfa. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.66:45183: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:49.846130  8021 raft_consensus.cc:2272] T 00f1ed4b42dc4bb4b840ad717387a021 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.848245  8021 tablet_replica.cc:333] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.848829  8021 raft_consensus.cc:2243] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.849269  8021 raft_consensus.cc:2272] T 9775f70e55ab4ab39d839e01d281256a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.851039  8021 tablet_replica.cc:333] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.851614  8021 raft_consensus.cc:2243] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.852146  8021 raft_consensus.cc:2272] T c77c97e9a68a44b5aaef74caedd8440a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.853955  8021 tablet_replica.cc:333] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.854516  8021 raft_consensus.cc:2243] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.854954  8021 raft_consensus.cc:2272] T 8168a7e324904bdca04258a02582133e P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.856762  8021 tablet_replica.cc:333] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.857321  8021 raft_consensus.cc:2243] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.857771  8021 raft_consensus.cc:2272] T b465daf7d17e4407b2226ba37042529c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.859679  8021 tablet_replica.cc:333] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.860304  8021 raft_consensus.cc:2243] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.861093  8021 raft_consensus.cc:2272] T 6885c0e03a4a403d9cdaad7803be4963 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.862903  8021 tablet_replica.cc:333] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.863514  8021 raft_consensus.cc:2243] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.864048  8021 raft_consensus.cc:2272] T fa0f514d6ce1487ab6000263ade7054c P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.865963  8021 tablet_replica.cc:333] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.866510  8021 raft_consensus.cc:2243] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.866933  8021 raft_consensus.cc:2272] T 930abb26f64e42619494dba489c18df1 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.868865  8021 tablet_replica.cc:333] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.869387  8021 raft_consensus.cc:2243] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.869798  8021 raft_consensus.cc:2272] T 522a49a4e17842f9b78405b0a23b9ba9 P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.871548  8021 tablet_replica.cc:333] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.872129  8021 raft_consensus.cc:2243] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.872593  8021 raft_consensus.cc:2272] T 0bda43ca086543e7b9665d96bdd452fa P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.874478  8021 tablet_replica.cc:333] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.875016  8021 raft_consensus.cc:2243] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.875438  8021 raft_consensus.cc:2272] T ac609b21527a4972b105bdb06f4caaca P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.877393  8021 tablet_replica.cc:333] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.877931  8021 raft_consensus.cc:2243] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.878671  8021 raft_consensus.cc:2272] T a6f9c7fa6f8b4a93affd76594fe4ed6e P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.880610  8021 tablet_replica.cc:333] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.881170  8021 raft_consensus.cc:2243] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.881595  8021 raft_consensus.cc:2272] T 01c74d4f7b884ba68067c533661fa56a P c15cc53a71664205bbbbc284c1d299ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.883298  8021 tablet_replica.cc:333] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.883846  8021 raft_consensus.cc:2243] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.884564  8021 raft_consensus.cc:2272] T b2328b3b8dd64a41b451825e2156884b P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.886369  8021 tablet_replica.cc:333] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.886839  8021 raft_consensus.cc:2243] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.887544  8021 raft_consensus.cc:2272] T d8e87107984c45c3aa20aed61a040a35 P c15cc53a71664205bbbbc284c1d299ff [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.889386  8021 tablet_replica.cc:333] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff: stopping tablet replica
I20260709 13:07:49.889842  8021 raft_consensus.cc:2243] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.890254  8021 raft_consensus.cc:2272] T d3c63b593e614ba995254f08593ea75b P c15cc53a71664205bbbbc284c1d299ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.929567  8021 tablet_server.cc:196] TabletServer@127.7.213.67:0 shutdown complete.
I20260709 13:07:49.961557  8021 master.cc:562] Master@127.7.213.126:34049 shutting down...
I20260709 13:07:49.984400  8021 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:49.984979  8021 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.985307  8021 tablet_replica.cc:333] T 00000000000000000000000000000000 P d0993257ed3749988dc227570cae9a8b: stopping tablet replica
I20260709 13:07:50.007664  8021 master.cc:584] Master@127.7.213.126:34049 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (14563 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:07:50.059958  8021 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.7.213.126:33533
I20260709 13:07:50.061221  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:50.066996 18901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:50.067078 18902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:50.068917 18904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:50.069427  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:50.070467  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:50.070689  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:50.070871  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602470070854 us; error 0 us; skew 500 ppm
I20260709 13:07:50.071374  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:50.073880  8021 webserver.cc:533] Webserver started at http://127.7.213.126:34753/ using document root <none> and password file <none>
I20260709 13:07:50.074373  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:50.074589  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:50.074862  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:50.076045  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/master-0-root/instance:
uuid: "2acd56cd48af490797ad208fbb71b121"
format_stamp: "Formatted at 2026-07-09 13:07:50 on dist-test-slave-9g29"
I20260709 13:07:50.080442  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:07:50.084136 18909 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.085119  8021 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:50.085378  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/master-0-root
uuid: "2acd56cd48af490797ad208fbb71b121"
format_stamp: "Formatted at 2026-07-09 13:07:50 on dist-test-slave-9g29"
I20260709 13:07:50.085609  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:50.116101  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:50.117177  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:50.155239  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.126:33533
I20260709 13:07:50.155371 18960 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.126:33533 every 8 connection(s)
I20260709 13:07:50.159447 18961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.170208 18961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121: Bootstrap starting.
I20260709 13:07:50.174894 18961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.179131 18961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121: No bootstrap required, opened a new log
I20260709 13:07:50.181380 18961 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd56cd48af490797ad208fbb71b121" member_type: VOTER }
I20260709 13:07:50.181788 18961 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:50.182041 18961 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2acd56cd48af490797ad208fbb71b121, State: Initialized, Role: FOLLOWER
I20260709 13:07:50.182631 18961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd56cd48af490797ad208fbb71b121" member_type: VOTER }
I20260709 13:07:50.183058 18961 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:50.183297 18961 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:50.183537 18961 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:50.188357 18961 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd56cd48af490797ad208fbb71b121" member_type: VOTER }
I20260709 13:07:50.188900 18961 leader_election.cc:304] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2acd56cd48af490797ad208fbb71b121; no voters: 
I20260709 13:07:50.190059 18961 leader_election.cc:290] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:50.190404 18964 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:50.191890 18964 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 1 LEADER]: Becoming Leader. State: Replica: 2acd56cd48af490797ad208fbb71b121, State: Running, Role: LEADER
I20260709 13:07:50.192598 18964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd56cd48af490797ad208fbb71b121" member_type: VOTER }
I20260709 13:07:50.192911 18961 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:50.197132 18966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2acd56cd48af490797ad208fbb71b121. Latest consensus state: current_term: 1 leader_uuid: "2acd56cd48af490797ad208fbb71b121" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd56cd48af490797ad208fbb71b121" member_type: VOTER } }
I20260709 13:07:50.197888 18966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:50.198221 18965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2acd56cd48af490797ad208fbb71b121" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd56cd48af490797ad208fbb71b121" member_type: VOTER } }
I20260709 13:07:50.198781 18965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:50.200109 18972 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:50.206324 18972 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:50.207274  8021 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:50.215175 18972 catalog_manager.cc:1383] Generated new cluster ID: 444d47a30e1247d09750f4b94697e03a
I20260709 13:07:50.215476 18972 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:50.226956 18972 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:50.228354 18972 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:50.244508 18972 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121: Generated new TSK 0
I20260709 13:07:50.245221 18972 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:50.274435  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:50.280884 18982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:50.281754 18983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:50.287469  8021 server_base.cc:1061] running on GCE node
W20260709 13:07:50.287492 18985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:50.288949  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:50.289139  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:50.289263  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602470289252 us; error 0 us; skew 500 ppm
I20260709 13:07:50.289707  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:50.292001  8021 webserver.cc:533] Webserver started at http://127.7.213.65:40661/ using document root <none> and password file <none>
I20260709 13:07:50.292420  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:50.292567  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:50.292783  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:50.293859  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-0-root/instance:
uuid: "522cf294e90d4bf18c109548cda61152"
format_stamp: "Formatted at 2026-07-09 13:07:50 on dist-test-slave-9g29"
I20260709 13:07:50.298367  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:07:50.301762 18990 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.302719  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:50.302965  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-0-root
uuid: "522cf294e90d4bf18c109548cda61152"
format_stamp: "Formatted at 2026-07-09 13:07:50 on dist-test-slave-9g29"
I20260709 13:07:50.303203  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:50.313730  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:50.314937  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:50.316533  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:50.319001  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:50.319193  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.319491  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:50.319670  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.363016  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.65:43671
I20260709 13:07:50.363147 19052 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.65:43671 every 8 connection(s)
I20260709 13:07:50.368120  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:50.375763 19057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:50.376931 19058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:50.379150 19060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:50.380048  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:50.381572 19053 heartbeater.cc:344] Connected to a master server at 127.7.213.126:33533
I20260709 13:07:50.381866  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:07:50.381983 19053 heartbeater.cc:461] Registering TS with master...
W20260709 13:07:50.382162  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:50.382421  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602470382403 us; error 0 us; skew 500 ppm
I20260709 13:07:50.382926 19053 heartbeater.cc:507] Master 127.7.213.126:33533 requested a full tablet report, sending...
I20260709 13:07:50.383013  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:50.385221 18926 ts_manager.cc:194] Registered new tserver with Master: 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671)
I20260709 13:07:50.385795  8021 webserver.cc:533] Webserver started at http://127.7.213.66:36321/ using document root <none> and password file <none>
I20260709 13:07:50.386341  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:50.386549  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:50.386827  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:50.387393 18926 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52250
I20260709 13:07:50.388109  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-1-root/instance:
uuid: "47483c02b1e348278a40781fdc137f96"
format_stamp: "Formatted at 2026-07-09 13:07:50 on dist-test-slave-9g29"
I20260709 13:07:50.392867  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:07:50.396590 19065 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.397380  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:50.397729  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-1-root
uuid: "47483c02b1e348278a40781fdc137f96"
format_stamp: "Formatted at 2026-07-09 13:07:50 on dist-test-slave-9g29"
I20260709 13:07:50.398041  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:50.422040  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:50.423324  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:50.424893  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:50.427270  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:50.427518  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.427922  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:50.428140  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.470467  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.66:40053
I20260709 13:07:50.470567 19127 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.66:40053 every 8 connection(s)
I20260709 13:07:50.475630  8021 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:50.483081 19131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:50.485495 19132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:50.487172 19128 heartbeater.cc:344] Connected to a master server at 127.7.213.126:33533
I20260709 13:07:50.487811 19128 heartbeater.cc:461] Registering TS with master...
W20260709 13:07:50.488817 19134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:50.488976 19128 heartbeater.cc:507] Master 127.7.213.126:33533 requested a full tablet report, sending...
I20260709 13:07:50.489274  8021 server_base.cc:1061] running on GCE node
I20260709 13:07:50.490289  8021 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:50.490540  8021 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:50.490760  8021 hybrid_clock.cc:648] HybridClock initialized: now 1783602470490741 us; error 0 us; skew 500 ppm
I20260709 13:07:50.491371  8021 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:50.491935 18926 ts_manager.cc:194] Registered new tserver with Master: 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:50.493918 18926 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52254
I20260709 13:07:50.494202  8021 webserver.cc:533] Webserver started at http://127.7.213.67:46383/ using document root <none> and password file <none>
I20260709 13:07:50.494973  8021 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:50.495246  8021 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:50.495566  8021 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:50.496798  8021 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-2-root/instance:
uuid: "3f652e73eb8045f6bcdcfe70e5047960"
format_stamp: "Formatted at 2026-07-09 13:07:50 on dist-test-slave-9g29"
I20260709 13:07:50.501782  8021 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:07:50.505707 19139 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.506524  8021 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:07:50.506860  8021 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-2-root
uuid: "3f652e73eb8045f6bcdcfe70e5047960"
format_stamp: "Formatted at 2026-07-09 13:07:50 on dist-test-slave-9g29"
I20260709 13:07:50.507216  8021 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxWoGB8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602158212224-8021-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:50.516201  8021 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:50.517474  8021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:50.519093  8021 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:50.521561  8021 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:50.521809  8021 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.522079  8021 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:50.522281  8021 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:50.564524  8021 rpc_server.cc:307] RPC server started. Bound to: 127.7.213.67:36609
I20260709 13:07:50.564639 19201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.213.67:36609 every 8 connection(s)
I20260709 13:07:50.579382 19202 heartbeater.cc:344] Connected to a master server at 127.7.213.126:33533
I20260709 13:07:50.579911 19202 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:50.580619 19202 heartbeater.cc:507] Master 127.7.213.126:33533 requested a full tablet report, sending...
I20260709 13:07:50.582463 18926 ts_manager.cc:194] Registered new tserver with Master: 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:50.583009  8021 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015022512s
I20260709 13:07:50.584074 18926 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52268
I20260709 13:07:50.611047 18926 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52272:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:50.613806 18926 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:50.684190 19167 tablet_service.cc:1511] Processing CreateTablet for tablet c4400e1db38f4227bc7ddcde78d93a11 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:50.685997 19167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4400e1db38f4227bc7ddcde78d93a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.686173 19166 tablet_service.cc:1511] Processing CreateTablet for tablet 25af7d7302394721a33cd7017a84bea3 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:50.688323 19166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25af7d7302394721a33cd7017a84bea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.699350 19165 tablet_service.cc:1511] Processing CreateTablet for tablet a20d75390629460099ed0728baefb3ab (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:50.701129 19165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a20d75390629460099ed0728baefb3ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.706540 19164 tablet_service.cc:1511] Processing CreateTablet for tablet 1f35f40f03074dd4862bc0e80fd8caed (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:50.708353 19164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f35f40f03074dd4862bc0e80fd8caed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.715015 19091 tablet_service.cc:1511] Processing CreateTablet for tablet a20d75390629460099ed0728baefb3ab (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:50.716668 19091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a20d75390629460099ed0728baefb3ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.721107 19163 tablet_service.cc:1511] Processing CreateTablet for tablet c4c055085a1d4959a878054bb8fd614c (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:50.722673 19163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c055085a1d4959a878054bb8fd614c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.725049 19090 tablet_service.cc:1511] Processing CreateTablet for tablet 1f35f40f03074dd4862bc0e80fd8caed (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:50.726580 19090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f35f40f03074dd4862bc0e80fd8caed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.743535 19088 tablet_service.cc:1511] Processing CreateTablet for tablet 3016133635d74bcdbf0b50a9086778ab (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:50.744369 19011 tablet_service.cc:1511] Processing CreateTablet for tablet ed00071d9a064800985eeae64cdda3e7 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:50.745172 19088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3016133635d74bcdbf0b50a9086778ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.745860 19011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed00071d9a064800985eeae64cdda3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.728104 19089 tablet_service.cc:1511] Processing CreateTablet for tablet c4c055085a1d4959a878054bb8fd614c (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:50.749617 19089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c055085a1d4959a878054bb8fd614c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.749313 19087 tablet_service.cc:1511] Processing CreateTablet for tablet 0864698179204de6a0c3ab6f41959297 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:50.749723 19166 tablet_service.cc:1511] Processing CreateTablet for tablet ed00071d9a064800985eeae64cdda3e7 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:50.730252 19161 tablet_service.cc:1511] Processing CreateTablet for tablet 0864698179204de6a0c3ab6f41959297 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:50.750787 19087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0864698179204de6a0c3ab6f41959297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.751086 19166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed00071d9a064800985eeae64cdda3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.732394 19012 tablet_service.cc:1511] Processing CreateTablet for tablet 0864698179204de6a0c3ab6f41959297 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:50.752219 19161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0864698179204de6a0c3ab6f41959297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.752701 19012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0864698179204de6a0c3ab6f41959297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.755410 19010 tablet_service.cc:1511] Processing CreateTablet for tablet 39c48d1152f54851b1209458b609506e (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:50.757009 19010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39c48d1152f54851b1209458b609506e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.735833 19014 tablet_service.cc:1511] Processing CreateTablet for tablet c4c055085a1d4959a878054bb8fd614c (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:50.760804 19014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c055085a1d4959a878054bb8fd614c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.735833 19016 tablet_service.cc:1511] Processing CreateTablet for tablet a20d75390629460099ed0728baefb3ab (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:50.764793 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a20d75390629460099ed0728baefb3ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.734004 19013 tablet_service.cc:1511] Processing CreateTablet for tablet 3016133635d74bcdbf0b50a9086778ab (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:50.768755 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3016133635d74bcdbf0b50a9086778ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.737089 19015 tablet_service.cc:1511] Processing CreateTablet for tablet 1f35f40f03074dd4862bc0e80fd8caed (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:50.772712 19015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f35f40f03074dd4862bc0e80fd8caed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.738798 19092 tablet_service.cc:1511] Processing CreateTablet for tablet 25af7d7302394721a33cd7017a84bea3 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:50.740314 19018 tablet_service.cc:1511] Processing CreateTablet for tablet c4400e1db38f4227bc7ddcde78d93a11 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:50.777297 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4400e1db38f4227bc7ddcde78d93a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.740592 19162 tablet_service.cc:1511] Processing CreateTablet for tablet 3016133635d74bcdbf0b50a9086778ab (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:50.738929 19093 tablet_service.cc:1511] Processing CreateTablet for tablet c4400e1db38f4227bc7ddcde78d93a11 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:50.732276 19017 tablet_service.cc:1511] Processing CreateTablet for tablet 25af7d7302394721a33cd7017a84bea3 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:50.780697 19162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3016133635d74bcdbf0b50a9086778ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.781335 19017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25af7d7302394721a33cd7017a84bea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.785451 19086 tablet_service.cc:1511] Processing CreateTablet for tablet ed00071d9a064800985eeae64cdda3e7 (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:50.788659 19085 tablet_service.cc:1511] Processing CreateTablet for tablet 39c48d1152f54851b1209458b609506e (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:50.793098 19093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4400e1db38f4227bc7ddcde78d93a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.794890 19092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25af7d7302394721a33cd7017a84bea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.800258 19085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39c48d1152f54851b1209458b609506e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.809648 19086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed00071d9a064800985eeae64cdda3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.836628 19164 tablet_service.cc:1511] Processing CreateTablet for tablet 39c48d1152f54851b1209458b609506e (DEFAULT_TABLE table=test-workload [id=54bd55dbbb2b48159c6bdc27fb60e5f6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:50.838055 19164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39c48d1152f54851b1209458b609506e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.870312 19217 tablet_bootstrap.cc:492] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:50.888761 19218 tablet_bootstrap.cc:492] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:50.897439 19219 tablet_bootstrap.cc:492] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:50.908821 19217 tablet_bootstrap.cc:654] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.909365 19219 tablet_bootstrap.cc:654] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.914091 19218 tablet_bootstrap.cc:654] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.916103 19219 tablet_bootstrap.cc:492] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:50.916680 19219 ts_tablet_manager.cc:1403] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.000s
I20260709 13:07:50.919381 19217 tablet_bootstrap.cc:492] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:50.919982 19218 tablet_bootstrap.cc:492] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:50.920033 19217 ts_tablet_manager.cc:1403] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.000s
I20260709 13:07:50.920632 19218 ts_tablet_manager.cc:1403] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.005s
I20260709 13:07:50.920269 19219 raft_consensus.cc:359] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:50.921433 19219 raft_consensus.cc:385] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:50.921785 19219 raft_consensus.cc:740] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:50.922613 19219 consensus_queue.cc:260] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:50.923553 19217 raft_consensus.cc:359] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:07:50.924464 19217 raft_consensus.cc:385] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:50.924057 19218 raft_consensus.cc:359] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:50.924865 19217 raft_consensus.cc:740] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:50.925022 19218 raft_consensus.cc:385] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:50.925288 19053 heartbeater.cc:499] Master 127.7.213.126:33533 was elected leader, sending a full tablet report...
I20260709 13:07:50.925455 19218 raft_consensus.cc:740] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:50.925704 19219 ts_tablet_manager.cc:1434] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:07:50.925920 19217 consensus_queue.cc:260] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:07:50.926802 19219 tablet_bootstrap.cc:492] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:50.926393 19218 consensus_queue.cc:260] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:50.935909 19217 ts_tablet_manager.cc:1434] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 13:07:50.936074 19202 heartbeater.cc:499] Master 127.7.213.126:33533 was elected leader, sending a full tablet report...
I20260709 13:07:50.936477 19219 tablet_bootstrap.cc:654] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.941948 19217 tablet_bootstrap.cc:492] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:50.950234 19217 tablet_bootstrap.cc:654] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.952488 19128 heartbeater.cc:499] Master 127.7.213.126:33533 was elected leader, sending a full tablet report...
I20260709 13:07:50.953370 19218 ts_tablet_manager.cc:1434] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.032s	user 0.000s	sys 0.031s
I20260709 13:07:50.954766 19218 tablet_bootstrap.cc:492] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:50.956697 19219 tablet_bootstrap.cc:492] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:50.957322 19219 ts_tablet_manager.cc:1403] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.004s
I20260709 13:07:50.962615 19218 tablet_bootstrap.cc:654] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.963102 19224 raft_consensus.cc:493] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.963698 19224 raft_consensus.cc:515] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:07:50.962018 19219 raft_consensus.cc:359] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:50.966128 19219 raft_consensus.cc:385] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:50.967231 19219 raft_consensus.cc:740] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:50.969599 19219 consensus_queue.cc:260] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:50.971596 19219 ts_tablet_manager.cc:1434] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.003s
I20260709 13:07:50.971783 19225 raft_consensus.cc:493] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.972708 19219 tablet_bootstrap.cc:492] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:50.972564 19225 raft_consensus.cc:515] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
W20260709 13:07:50.975167 19129 tablet.cc:2406] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:50.981403 19217 tablet_bootstrap.cc:492] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:50.982084 19217 ts_tablet_manager.cc:1403] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.005s
I20260709 13:07:50.983521 19219 tablet_bootstrap.cc:654] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:50.985425 19217 raft_consensus.cc:359] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:50.986970 19223 raft_consensus.cc:493] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:50.987651 19223 raft_consensus.cc:515] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.005376 19224 leader_election.cc:290] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:51.007195 19223 leader_election.cc:290] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.017063 19225 leader_election.cc:290] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.017536 19217 raft_consensus.cc:385] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.018380 19217 raft_consensus.cc:740] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.023600 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:51.023773 19217 consensus_queue.cc:260] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
W20260709 13:07:51.026513 19141 leader_election.cc:343] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.038878 19224 raft_consensus.cc:493] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.039731 19224 raft_consensus.cc:515] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.042356 19217 ts_tablet_manager.cc:1434] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.003s
I20260709 13:07:51.043558 19224 leader_election.cc:290] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:51.051887 19217 tablet_bootstrap.cc:492] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:51.086710 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f35f40f03074dd4862bc0e80fd8caed" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:51.113643 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
W20260709 13:07:51.113704 19141 leader_election.cc:343] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.122833 19219 tablet_bootstrap.cc:492] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
W20260709 13:07:51.124761 18994 leader_election.cc:343] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.129751 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.129657 19219 ts_tablet_manager.cc:1403] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.157s	user 0.027s	sys 0.054s
W20260709 13:07:51.132220 18991 leader_election.cc:343] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.132910 18991 leader_election.cc:304] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522cf294e90d4bf18c109548cda61152; no voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96
I20260709 13:07:51.135421 19223 raft_consensus.cc:2749] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:51.139395 19219 raft_consensus.cc:359] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:07:51.140782 19219 raft_consensus.cc:385] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.141250 19219 raft_consensus.cc:740] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.141625 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a20d75390629460099ed0728baefb3ab" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:51.142623 19219 consensus_queue.cc:260] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
W20260709 13:07:51.143723 19067 leader_election.cc:343] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.146067 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a20d75390629460099ed0728baefb3ab" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:51.147089 19219 ts_tablet_manager.cc:1434] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.001s
W20260709 13:07:51.147977 19069 leader_election.cc:343] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.148437 19218 tablet_bootstrap.cc:492] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:51.148665 19069 leader_election.cc:304] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47483c02b1e348278a40781fdc137f96; no voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:51.149554 19219 tablet_bootstrap.cc:492] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:51.150789 19225 raft_consensus.cc:2749] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:51.154795 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.155490 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f35f40f03074dd4862bc0e80fd8caed" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
W20260709 13:07:51.156293 19140 leader_election.cc:343] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.156930 19140 leader_election.cc:304] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960; no voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:51.157995 19224 raft_consensus.cc:2749] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:51.158900 19140 leader_election.cc:343] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.159422 19140 leader_election.cc:304] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960; no voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:51.159792 19219 tablet_bootstrap.cc:654] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.160921 19224 raft_consensus.cc:2749] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:51.149571 19218 ts_tablet_manager.cc:1403] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.195s	user 0.056s	sys 0.045s
I20260709 13:07:51.162683 19217 tablet_bootstrap.cc:654] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.169354 19217 tablet_bootstrap.cc:492] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:51.169898 19219 tablet_bootstrap.cc:492] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:51.170284 19217 ts_tablet_manager.cc:1403] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.127s	user 0.013s	sys 0.004s
I20260709 13:07:51.170639 19219 ts_tablet_manager.cc:1403] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.004s
I20260709 13:07:51.170135 19218 raft_consensus.cc:359] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.171221 19218 raft_consensus.cc:385] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.171589 19218 raft_consensus.cc:740] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.172581 19218 consensus_queue.cc:260] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.174171 19217 raft_consensus.cc:359] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.174566 19219 raft_consensus.cc:359] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.175170 19217 raft_consensus.cc:385] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.175557 19219 raft_consensus.cc:385] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.175678 19217 raft_consensus.cc:740] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.175783 19218 ts_tablet_manager.cc:1434] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:07:51.176083 19219 raft_consensus.cc:740] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.177299 19217 consensus_queue.cc:260] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.178021 19218 tablet_bootstrap.cc:492] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:51.177424 19219 consensus_queue.cc:260] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.182943 19219 ts_tablet_manager.cc:1434] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:07:51.184692 19219 tablet_bootstrap.cc:492] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:51.187484 19218 tablet_bootstrap.cc:654] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.188299 19217 ts_tablet_manager.cc:1434] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:07:51.191211 19217 tablet_bootstrap.cc:492] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:51.198803 19217 tablet_bootstrap.cc:654] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.200075 19218 tablet_bootstrap.cc:492] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:51.200784 19218 ts_tablet_manager.cc:1403] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.008s
W20260709 13:07:51.203997 18974 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:51.204634 19218 raft_consensus.cc:359] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.205269 19219 tablet_bootstrap.cc:654] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.205500 19218 raft_consensus.cc:385] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.205869 19218 raft_consensus.cc:740] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.206914 19218 consensus_queue.cc:260] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.210043 19218 ts_tablet_manager.cc:1434] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:07:51.212047 19218 tablet_bootstrap.cc:492] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:51.217588 19219 tablet_bootstrap.cc:492] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:51.218272 19219 ts_tablet_manager.cc:1403] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.004s
I20260709 13:07:51.221035 19225 raft_consensus.cc:493] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.221632 19225 raft_consensus.cc:515] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.222154 19219 raft_consensus.cc:359] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.222900 19217 tablet_bootstrap.cc:492] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:51.223078 19219 raft_consensus.cc:385] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.223498 19219 raft_consensus.cc:740] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.223573 19217 ts_tablet_manager.cc:1403] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.015s
I20260709 13:07:51.224821 19218 tablet_bootstrap.cc:654] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.225653 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25af7d7302394721a33cd7017a84bea3" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:51.225787 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25af7d7302394721a33cd7017a84bea3" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:51.226557 19177 raft_consensus.cc:2468] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 0.
W20260709 13:07:51.227463 19067 leader_election.cc:343] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.227945 19217 raft_consensus.cc:359] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.228964 19217 raft_consensus.cc:385] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.229324 19217 raft_consensus.cc:740] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.229296 19219 consensus_queue.cc:260] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.230594 19069 leader_election.cc:304] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 522cf294e90d4bf18c109548cda61152
I20260709 13:07:51.231212 19218 tablet_bootstrap.cc:492] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:51.231724 19218 ts_tablet_manager.cc:1403] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.003s
I20260709 13:07:51.231662 19228 raft_consensus.cc:2804] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.232121 19219 ts_tablet_manager.cc:1434] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:07:51.232296 19228 raft_consensus.cc:493] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.232807 19228 raft_consensus.cc:3060] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.233958 19219 tablet_bootstrap.cc:492] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:51.230417 19217 consensus_queue.cc:260] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.235862 19218 raft_consensus.cc:359] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.236593 19217 ts_tablet_manager.cc:1434] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:07:51.236624 19218 raft_consensus.cc:385] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.237224 19218 raft_consensus.cc:740] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.238386 19217 tablet_bootstrap.cc:492] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:51.238273 19218 consensus_queue.cc:260] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.242923 19219 tablet_bootstrap.cc:654] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.247184 19225 leader_election.cc:290] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.247364 19217 tablet_bootstrap.cc:654] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.256464 19219 tablet_bootstrap.cc:492] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:51.257062 19219 ts_tablet_manager.cc:1403] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.010s
I20260709 13:07:51.260949 19218 ts_tablet_manager.cc:1434] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.017s
I20260709 13:07:51.262511 19218 tablet_bootstrap.cc:492] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:51.263002 19217 tablet_bootstrap.cc:492] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:51.263598 19217 ts_tablet_manager.cc:1403] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.005s
I20260709 13:07:51.265311 19223 raft_consensus.cc:493] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.265447 19219 raft_consensus.cc:359] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.265971 19223 raft_consensus.cc:515] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:07:51.266464 19219 raft_consensus.cc:385] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.266968 19219 raft_consensus.cc:740] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.267976 19219 consensus_queue.cc:260] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.269527 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:51.270309 19177 raft_consensus.cc:2468] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:51.270124 19228 raft_consensus.cc:515] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.270852 19218 tablet_bootstrap.cc:654] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.271781 18994 leader_election.cc:304] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.274819 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
W20260709 13:07:51.276345 18991 leader_election.cc:343] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.277010 19223 leader_election.cc:290] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:51.280699 19223 raft_consensus.cc:2804] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.281096 19223 raft_consensus.cc:493] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.281482 19223 raft_consensus.cc:3060] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.285270 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25af7d7302394721a33cd7017a84bea3" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:51.286209 19177 raft_consensus.cc:3060] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.287670 19217 raft_consensus.cc:359] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.288654 19217 raft_consensus.cc:385] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.289007 19217 raft_consensus.cc:740] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.294902 19228 leader_election.cc:290] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.297587 19217 consensus_queue.cc:260] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.298702 19177 raft_consensus.cc:2468] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 1.
I20260709 13:07:51.300519 19069 leader_election.cc:304] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:51.301980 19228 raft_consensus.cc:2804] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.303124 19217 ts_tablet_manager.cc:1434] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:07:51.303397 19228 raft_consensus.cc:697] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:51.304592 19228 consensus_queue.cc:237] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.309718 19223 raft_consensus.cc:515] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:07:51.310976 19217 tablet_bootstrap.cc:492] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:51.312522 19223 leader_election.cc:290] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:51.313763 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
W20260709 13:07:51.315296 18991 leader_election.cc:343] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.315984 19218 tablet_bootstrap.cc:492] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:51.316527 19218 ts_tablet_manager.cc:1403] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.032s
I20260709 13:07:51.317917 19219 ts_tablet_manager.cc:1434] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.060s	user 0.026s	sys 0.011s
I20260709 13:07:51.319324 19219 tablet_bootstrap.cc:492] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:51.319929 19218 raft_consensus.cc:359] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.320770 19218 raft_consensus.cc:385] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.321117 19218 raft_consensus.cc:740] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.322074 19218 consensus_queue.cc:260] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.325012 19218 ts_tablet_manager.cc:1434] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:51.327467 19218 tablet_bootstrap.cc:492] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:51.330757 19219 tablet_bootstrap.cc:654] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.335697 19218 tablet_bootstrap.cc:654] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.329715 19217 tablet_bootstrap.cc:654] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:51.339591 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:36609, user_credentials={real_user=slave}} blocked reactor thread for 67916.2us
I20260709 13:07:51.340746 18925 catalog_manager.cc:5697] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:51.344692 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:51.345582 19177 raft_consensus.cc:3060] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.346908 19249 raft_consensus.cc:493] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.347616 19248 raft_consensus.cc:493] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.348532 19249 raft_consensus.cc:515] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.348769 19248 raft_consensus.cc:515] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.352016 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3016133635d74bcdbf0b50a9086778ab" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.352276 19248 leader_election.cc:290] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
W20260709 13:07:51.354028 18991 leader_election.cc:343] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.354516 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0864698179204de6a0c3ab6f41959297" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.355371 19102 raft_consensus.cc:2468] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:51.356781 18991 leader_election.cc:304] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.357188 19219 tablet_bootstrap.cc:492] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:51.357685 19248 raft_consensus.cc:2804] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.358157 19248 raft_consensus.cc:493] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.358325 19219 ts_tablet_manager.cc:1403] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.014s
I20260709 13:07:51.358554 19248 raft_consensus.cc:3060] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.360584 19217 tablet_bootstrap.cc:492] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:51.361382 19217 ts_tablet_manager.cc:1403] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.057s	user 0.008s	sys 0.006s
I20260709 13:07:51.362295 19219 raft_consensus.cc:359] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.363224 19219 raft_consensus.cc:385] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.363555 19219 raft_consensus.cc:740] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.364373 19218 tablet_bootstrap.cc:492] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:51.364957 19218 ts_tablet_manager.cc:1403] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.003s
I20260709 13:07:51.364616 19219 consensus_queue.cc:260] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.366340 19217 raft_consensus.cc:359] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.367607 19217 raft_consensus.cc:385] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.368180 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25af7d7302394721a33cd7017a84bea3" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:51.368230 19217 raft_consensus.cc:740] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.368953 19028 raft_consensus.cc:3060] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.369711 19217 consensus_queue.cc:260] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.372270 19219 ts_tablet_manager.cc:1434] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 13:07:51.373457 19217 ts_tablet_manager.cc:1434] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.010s
I20260709 13:07:51.373699 19219 tablet_bootstrap.cc:492] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:51.374984 19217 tablet_bootstrap.cc:492] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:51.382292 19219 tablet_bootstrap.cc:654] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.383411 19217 tablet_bootstrap.cc:654] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.389747 19176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0864698179204de6a0c3ab6f41959297" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
W20260709 13:07:51.392108 18994 leader_election.cc:343] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.394538 19249 leader_election.cc:290] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.357227 19177 raft_consensus.cc:2468] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:51.397681 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3016133635d74bcdbf0b50a9086778ab" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:51.398576 19177 raft_consensus.cc:2468] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:51.399829 19028 raft_consensus.cc:2468] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 1.
I20260709 13:07:51.401154 18994 leader_election.cc:304] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 47483c02b1e348278a40781fdc137f96
I20260709 13:07:51.402153 19249 raft_consensus.cc:2804] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.402649 19249 raft_consensus.cc:697] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:51.403808 19219 tablet_bootstrap.cc:492] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:51.403512 19249 consensus_queue.cc:237] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:07:51.404520 19219 ts_tablet_manager.cc:1403] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.008s
I20260709 13:07:51.405175 19217 tablet_bootstrap.cc:492] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:51.405967 19217 ts_tablet_manager.cc:1403] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.009s
I20260709 13:07:51.407696 18994 leader_election.cc:304] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 47483c02b1e348278a40781fdc137f96
I20260709 13:07:51.408594 19223 raft_consensus.cc:2804] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.408951 19223 raft_consensus.cc:493] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.409286 19223 raft_consensus.cc:3060] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.410197 19217 raft_consensus.cc:359] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.411058 19217 raft_consensus.cc:385] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.411379 19217 raft_consensus.cc:740] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.412283 19248 raft_consensus.cc:515] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.412765 19217 consensus_queue.cc:260] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.414546 19248 leader_election.cc:290] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.415549 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0864698179204de6a0c3ab6f41959297" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:51.416194 19102 raft_consensus.cc:3060] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.417079 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0864698179204de6a0c3ab6f41959297" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
W20260709 13:07:51.418862 18994 leader_election.cc:343] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.419049 19217 ts_tablet_manager.cc:1434] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.003s
I20260709 13:07:51.420667 19217 tablet_bootstrap.cc:492] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:51.423763 19228 raft_consensus.cc:493] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.424526 19228 raft_consensus.cc:515] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.428891 19217 tablet_bootstrap.cc:654] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.415573 19218 raft_consensus.cc:359] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.407971 19219 raft_consensus.cc:359] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.430878 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c48d1152f54851b1209458b609506e" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:51.432452 19228 leader_election.cc:290] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.433898 19219 raft_consensus.cc:385] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.434253 19219 raft_consensus.cc:740] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.432380 19218 raft_consensus.cc:385] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.435381 19102 raft_consensus.cc:2468] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:51.435416 19219 consensus_queue.cc:260] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.436730 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c48d1152f54851b1209458b609506e" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:51.428488 19223 raft_consensus.cc:515] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.437052 18991 leader_election.cc:304] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 3f652e73eb8045f6bcdcfe70e5047960
W20260709 13:07:51.438284 19069 leader_election.cc:343] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:51.439437 19067 leader_election.cc:343] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:51.439889 19067 leader_election.cc:304] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47483c02b1e348278a40781fdc137f96; no voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:51.440078 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3016133635d74bcdbf0b50a9086778ab" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:51.441000 19228 raft_consensus.cc:2749] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:51.441632 19223 leader_election.cc:290] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.442907 19223 raft_consensus.cc:2804] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.443004 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3016133635d74bcdbf0b50a9086778ab" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:51.442376 19219 ts_tablet_manager.cc:1434] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:07:51.443604 19177 raft_consensus.cc:3060] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.435604 19218 raft_consensus.cc:740] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.444737 19217 tablet_bootstrap.cc:492] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:51.444631 19218 consensus_queue.cc:260] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.445287 19217 ts_tablet_manager.cc:1403] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.002s
I20260709 13:07:51.447331 19218 ts_tablet_manager.cc:1434] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.000s
I20260709 13:07:51.448565 19217 raft_consensus.cc:359] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.449498 19217 raft_consensus.cc:385] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.449851 19217 raft_consensus.cc:740] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.448061 19223 raft_consensus.cc:697] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:51.450529 19218 tablet_bootstrap.cc:492] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:51.447692 19102 raft_consensus.cc:3060] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.450060 18924 catalog_manager.cc:5697] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:51.451501 19223 consensus_queue.cc:237] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.452800 19177 raft_consensus.cc:2468] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:51.454306 18994 leader_election.cc:304] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.448968 19219 tablet_bootstrap.cc:492] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:51.455236 19248 raft_consensus.cc:2804] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.455716 19248 raft_consensus.cc:697] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:51.455794 19217 consensus_queue.cc:260] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.456610 19248 consensus_queue.cc:237] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.460093 19224 raft_consensus.cc:493] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.460796 19224 raft_consensus.cc:515] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.464967 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a20d75390629460099ed0728baefb3ab" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:51.465608 19224 leader_election.cc:290] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:51.465752 19028 raft_consensus.cc:2468] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:51.466226 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a20d75390629460099ed0728baefb3ab" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.466871 19103 raft_consensus.cc:2468] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:51.467559 19217 ts_tablet_manager.cc:1434] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:07:51.467347 19141 leader_election.cc:304] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.468794 19224 raft_consensus.cc:2804] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.469203 19224 raft_consensus.cc:493] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.469602 19224 raft_consensus.cc:3060] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.474588 19217 tablet_bootstrap.cc:492] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:51.478188 19224 raft_consensus.cc:515] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.478654 19102 raft_consensus.cc:2468] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:51.480471 19217 tablet_bootstrap.cc:654] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.481339 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a20d75390629460099ed0728baefb3ab" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:51.481956 19028 raft_consensus.cc:3060] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.488792 19218 tablet_bootstrap.cc:654] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.490068 19028 raft_consensus.cc:2468] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 1.
I20260709 13:07:51.490490 18924 catalog_manager.cc:5697] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:51.492458 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a20d75390629460099ed0728baefb3ab" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:51.494671 19103 raft_consensus.cc:3060] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.495795 18924 catalog_manager.cc:5697] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:51.494380 19219 tablet_bootstrap.cc:654] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.493847 19141 leader_election.cc:304] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.510159 19224 leader_election.cc:290] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:51.515156 19224 raft_consensus.cc:2804] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.516335 19224 raft_consensus.cc:697] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Becoming Leader. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:07:51.518316 19224 consensus_queue.cc:237] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.513844 19103 raft_consensus.cc:2468] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 1.
I20260709 13:07:51.533196 19217 tablet_bootstrap.cc:492] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:51.533517 19218 tablet_bootstrap.cc:492] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:51.533985 19217 ts_tablet_manager.cc:1403] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.060s	user 0.030s	sys 0.023s
I20260709 13:07:51.534536 19218 ts_tablet_manager.cc:1403] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.084s	user 0.007s	sys 0.019s
I20260709 13:07:51.537389 19217 raft_consensus.cc:359] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.538005 19217 raft_consensus.cc:385] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.538165 19217 raft_consensus.cc:740] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.538750 19217 consensus_queue.cc:260] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.540653 19217 ts_tablet_manager.cc:1434] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.006s
I20260709 13:07:51.540318 19218 raft_consensus.cc:359] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.541317 19218 raft_consensus.cc:385] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.541673 19218 raft_consensus.cc:740] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.541972 19219 tablet_bootstrap.cc:492] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:51.542599 19219 ts_tablet_manager.cc:1403] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.094s	user 0.014s	sys 0.012s
I20260709 13:07:51.542541 19218 consensus_queue.cc:260] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.545569 19218 ts_tablet_manager.cc:1434] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:07:51.545425 19219 raft_consensus.cc:359] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.546176 19219 raft_consensus.cc:385] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.546471 19219 raft_consensus.cc:740] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.547061 19218 tablet_bootstrap.cc:492] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:51.547261 19219 consensus_queue.cc:260] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.543885 18924 catalog_manager.cc:5697] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:51.552397 19219 ts_tablet_manager.cc:1434] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:07:51.555393 19218 tablet_bootstrap.cc:654] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.570122 19218 tablet_bootstrap.cc:492] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:51.570677 19218 ts_tablet_manager.cc:1403] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.005s
I20260709 13:07:51.573593 19218 raft_consensus.cc:359] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.574313 19218 raft_consensus.cc:385] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.574508 19218 raft_consensus.cc:740] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.575021 19218 consensus_queue.cc:260] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.576633 19218 ts_tablet_manager.cc:1434] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:07:51.577557 19218 tablet_bootstrap.cc:492] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:51.583853 19218 tablet_bootstrap.cc:654] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:51.589075 19218 tablet_bootstrap.cc:492] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:51.589624 19218 ts_tablet_manager.cc:1403] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:07:51.591869 19218 raft_consensus.cc:359] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:07:51.592360 19218 raft_consensus.cc:385] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:51.592540 19218 raft_consensus.cc:740] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:51.593173 19218 consensus_queue.cc:260] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:07:51.594921 19218 ts_tablet_manager.cc:1434] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:51.637038 19228 raft_consensus.cc:493] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.637734 19228 raft_consensus.cc:515] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.639590 19228 leader_election.cc:290] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.640208 19223 raft_consensus.cc:493] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.640571 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:51.640895 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:51.640765 19223 raft_consensus.cc:515] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.641772 19177 raft_consensus.cc:2468] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 0.
I20260709 13:07:51.642052 19028 raft_consensus.cc:2468] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 0.
I20260709 13:07:51.643061 19069 leader_election.cc:304] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:51.643366 19223 leader_election.cc:290] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.644743 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:51.644779 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.645375 19228 raft_consensus.cc:2804] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.645394 19177 raft_consensus.cc:2468] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:51.645908 19228 raft_consensus.cc:493] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.646314 19228 raft_consensus.cc:3060] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.646715 18994 leader_election.cc:304] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.647485 19223 raft_consensus.cc:2804] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.647823 19223 raft_consensus.cc:493] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.648134 19223 raft_consensus.cc:3060] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.652520 19228 raft_consensus.cc:515] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.653797 19103 raft_consensus.cc:2393] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522cf294e90d4bf18c109548cda61152 in current term 1: Already voted for candidate 47483c02b1e348278a40781fdc137f96 in this term.
I20260709 13:07:51.654008 19223 raft_consensus.cc:515] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.655587 19228 leader_election.cc:290] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.655815 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:51.656771 19028 raft_consensus.cc:2393] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 47483c02b1e348278a40781fdc137f96 in current term 1: Already voted for candidate 522cf294e90d4bf18c109548cda61152 in this term.
I20260709 13:07:51.657166 19223 leader_election.cc:290] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.657347 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:51.657850 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:51.658037 19177 raft_consensus.cc:3060] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.658458 19176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:51.658684 19103 raft_consensus.cc:2393] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 522cf294e90d4bf18c109548cda61152 in current term 1: Already voted for candidate 47483c02b1e348278a40781fdc137f96 in this term.
I20260709 13:07:51.660116 18991 leader_election.cc:304] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522cf294e90d4bf18c109548cda61152; no voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96
I20260709 13:07:51.661005 19223 raft_consensus.cc:2749] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:51.664572 19177 raft_consensus.cc:2468] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 1.
I20260709 13:07:51.665906 19069 leader_election.cc:304] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 522cf294e90d4bf18c109548cda61152
I20260709 13:07:51.666637 19228 raft_consensus.cc:2804] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.667131 19228 raft_consensus.cc:697] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:51.667865 19228 consensus_queue.cc:237] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.671727 19224 raft_consensus.cc:493] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.672384 19224 raft_consensus.cc:515] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.674301 19224 leader_election.cc:290] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:51.675123 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:51.675287 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.675941 19103 raft_consensus.cc:2468] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:51.675887 19028 raft_consensus.cc:2468] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:51.676244 18924 catalog_manager.cc:5697] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:51.677099 19140 leader_election.cc:304] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:51.677901 19224 raft_consensus.cc:2804] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.678215 19224 raft_consensus.cc:493] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.678484 19224 raft_consensus.cc:3060] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.684736 19224 raft_consensus.cc:515] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.687160 19224 leader_election.cc:290] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:51.688045 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:51.688300 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:51.688736 19028 raft_consensus.cc:3060] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.688839 19103 raft_consensus.cc:3060] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.696163 19103 raft_consensus.cc:2468] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 1.
I20260709 13:07:51.696163 19028 raft_consensus.cc:2468] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 1.
I20260709 13:07:51.697293 19140 leader_election.cc:304] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:51.698015 19224 raft_consensus.cc:2804] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.698459 19224 raft_consensus.cc:697] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Becoming Leader. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:07:51.699301 19224 consensus_queue.cc:237] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.706622 18924 catalog_manager.cc:5697] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:51.720724 19223 raft_consensus.cc:493] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.721474 19223 raft_consensus.cc:515] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.723186 19223 leader_election.cc:290] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.724093 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f35f40f03074dd4862bc0e80fd8caed" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.724396 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f35f40f03074dd4862bc0e80fd8caed" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:51.724821 19103 raft_consensus.cc:2468] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:51.725015 19177 raft_consensus.cc:2468] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:51.725886 18991 leader_election.cc:304] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.726590 19223 raft_consensus.cc:2804] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.726954 19223 raft_consensus.cc:493] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.727192 19223 raft_consensus.cc:3060] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.732827 19223 raft_consensus.cc:515] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.734428 19223 leader_election.cc:290] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.735357 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f35f40f03074dd4862bc0e80fd8caed" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:51.735579 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f35f40f03074dd4862bc0e80fd8caed" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:51.735925 19103 raft_consensus.cc:3060] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.736244 19177 raft_consensus.cc:3060] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.741043 19103 raft_consensus.cc:2468] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:51.742151 19177 raft_consensus.cc:2468] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:51.742244 18991 leader_election.cc:304] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.743216 19223 raft_consensus.cc:2804] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.743746 19223 raft_consensus.cc:697] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:51.744514 19223 consensus_queue.cc:237] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.747134 19248 raft_consensus.cc:493] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.747619 19248 raft_consensus.cc:515] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.750057 19248 leader_election.cc:290] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.750833 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c48d1152f54851b1209458b609506e" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.751044 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c48d1152f54851b1209458b609506e" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:51.751545 19103 raft_consensus.cc:2468] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:51.751684 19177 raft_consensus.cc:2468] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:51.752727 18991 leader_election.cc:304] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.753525 19248 raft_consensus.cc:2804] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:51.753924 19248 raft_consensus.cc:493] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:51.753230 18924 catalog_manager.cc:5697] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:51.754236 19248 raft_consensus.cc:3060] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.759915 19248 raft_consensus.cc:515] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.762158 19248 leader_election.cc:290] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:51.763319 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c48d1152f54851b1209458b609506e" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:51.763635 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c48d1152f54851b1209458b609506e" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:51.763932 19103 raft_consensus.cc:3060] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.764274 19177 raft_consensus.cc:3060] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.769160 19103 raft_consensus.cc:2468] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:51.769398 19177 raft_consensus.cc:2468] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:51.770274 18991 leader_election.cc:304] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:51.771142 19248 raft_consensus.cc:2804] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:51.771564 19248 raft_consensus.cc:697] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:51.772401 19248 consensus_queue.cc:237] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.779246 18924 catalog_manager.cc:5697] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:51.789917 19103 raft_consensus.cc:3060] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.796443 19248 consensus_queue.cc:1048] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:51.809443 19248 consensus_queue.cc:1048] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:51.819415 19224 raft_consensus.cc:493] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:51.819936 19224 raft_consensus.cc:515] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:51.822209 19224 leader_election.cc:290] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:51.822775 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0864698179204de6a0c3ab6f41959297" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:51.823374 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0864698179204de6a0c3ab6f41959297" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:51.824214 19103 raft_consensus.cc:2393] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f652e73eb8045f6bcdcfe70e5047960 in current term 1: Already voted for candidate 522cf294e90d4bf18c109548cda61152 in this term.
I20260709 13:07:51.825166 19140 leader_election.cc:304] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960; no voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:51.825887 19224 raft_consensus.cc:3060] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:51.832032 19224 raft_consensus.cc:2749] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:51.836323 19228 consensus_queue.cc:1048] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:51.847671 19228 consensus_queue.cc:1048] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:51.870848 19248 consensus_queue.cc:1048] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:51.881374 19248 consensus_queue.cc:1048] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:51.935386 19103 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "25af7d7302394721a33cd7017a84bea3"
dest_uuid: "47483c02b1e348278a40781fdc137f96"
mode: GRACEFUL
new_leader_uuid: "522cf294e90d4bf18c109548cda61152"
 from {username='slave'} at 127.0.0.1:36088
I20260709 13:07:51.936041 19103 raft_consensus.cc:606] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Received request to transfer leadership to TS 522cf294e90d4bf18c109548cda61152
I20260709 13:07:51.951026 19177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a20d75390629460099ed0728baefb3ab"
dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
mode: GRACEFUL
new_leader_uuid: "522cf294e90d4bf18c109548cda61152"
 from {username='slave'} at 127.0.0.1:56680
I20260709 13:07:51.951593 19177 raft_consensus.cc:606] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Received request to transfer leadership to TS 522cf294e90d4bf18c109548cda61152
I20260709 13:07:51.960737 19177 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c"
dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
mode: GRACEFUL
new_leader_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:56680
I20260709 13:07:51.961320 19177 raft_consensus.cc:606] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Received request to transfer leadership to TS 47483c02b1e348278a40781fdc137f96
I20260709 13:07:51.975504 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3016133635d74bcdbf0b50a9086778ab"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
mode: GRACEFUL
new_leader_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:07:51.976317 19028 raft_consensus.cc:606] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Received request to transfer leadership to TS 47483c02b1e348278a40781fdc137f96
I20260709 13:07:51.981853 19224 consensus_queue.cc:1048] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:51.982236 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0864698179204de6a0c3ab6f41959297"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
mode: GRACEFUL
new_leader_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:07:51.982831 19028 raft_consensus.cc:606] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Received request to transfer leadership to TS 47483c02b1e348278a40781fdc137f96
I20260709 13:07:51.990520 19224 raft_consensus.cc:993] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960: : Instructing follower 522cf294e90d4bf18c109548cda61152 to start an election
I20260709 13:07:51.990983 19224 raft_consensus.cc:1081] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Signalling peer 522cf294e90d4bf18c109548cda61152 to start an election
I20260709 13:07:51.993275 19027 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a20d75390629460099ed0728baefb3ab"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
 from {username='slave'} at 127.0.0.1:51452
I20260709 13:07:51.993953 19027 raft_consensus.cc:493] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:51.994333 19027 raft_consensus.cc:3060] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:51.996243 19265 consensus_queue.cc:1048] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:52.003369 19026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39c48d1152f54851b1209458b609506e"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
mode: GRACEFUL
new_leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:07:52.004531 19026 raft_consensus.cc:606] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Received request to transfer leadership to TS 3f652e73eb8045f6bcdcfe70e5047960
I20260709 13:07:52.003734 19248 consensus_queue.cc:1048] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:52.011575 19027 raft_consensus.cc:515] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.013329 19028 raft_consensus.cc:1240] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Rejecting Update request from peer 3f652e73eb8045f6bcdcfe70e5047960 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:52.014747 19027 leader_election.cc:290] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:52.016837 19101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a20d75390629460099ed0728baefb3ab" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:52.019475 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a20d75390629460099ed0728baefb3ab" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:52.020222 19177 raft_consensus.cc:3055] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:52.020581 19177 raft_consensus.cc:740] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:07:52.021543 19177 consensus_queue.cc:260] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.022488 19177 raft_consensus.cc:3060] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.024936 19249 raft_consensus.cc:993] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152: : Instructing follower 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:52.025534 19223 raft_consensus.cc:1081] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Signalling peer 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:52.031375 19249 consensus_queue.cc:1048] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:52.033691 19102 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0864698179204de6a0c3ab6f41959297"
dest_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:58076
I20260709 13:07:52.034364 19102 raft_consensus.cc:493] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:52.034758 19102 raft_consensus.cc:3060] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.037725 19177 raft_consensus.cc:2468] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 2.
I20260709 13:07:52.039842 18994 leader_election.cc:304] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 47483c02b1e348278a40781fdc137f96
I20260709 13:07:52.041806 19102 raft_consensus.cc:515] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.042444 19249 raft_consensus.cc:2804] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:52.045246 19102 leader_election.cc:290] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:52.045666 19175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0864698179204de6a0c3ab6f41959297" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:52.045711 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0864698179204de6a0c3ab6f41959297" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:52.045820 19249 raft_consensus.cc:697] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:52.046710 19028 raft_consensus.cc:3055] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:52.047142 19028 raft_consensus.cc:740] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:52.047582 19249 consensus_queue.cc:237] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.047930 19028 consensus_queue.cc:260] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.049240 19028 raft_consensus.cc:3060] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.057292 19028 raft_consensus.cc:2468] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 2.
I20260709 13:07:52.057657 18925 catalog_manager.cc:5697] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 1 to 2, leader changed from 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67) to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.059062 19067 leader_election.cc:304] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 3f652e73eb8045f6bcdcfe70e5047960
I20260709 13:07:52.060360 19228 raft_consensus.cc:2804] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:52.060871 19228 raft_consensus.cc:697] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:52.061695 19228 consensus_queue.cc:237] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.070163 18925 catalog_manager.cc:5697] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 1 to 2, leader changed from 522cf294e90d4bf18c109548cda61152 (127.7.213.65) to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.080228 19228 consensus_queue.cc:1048] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:52.095064 19245 consensus_queue.cc:1048] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:52.123369 19291 consensus_queue.cc:1048] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:52.127530 19249 consensus_queue.cc:1048] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:52.136530 19224 consensus_queue.cc:1048] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:52.147709 19249 consensus_queue.cc:1048] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:52.154145 19224 raft_consensus.cc:993] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960: : Instructing follower 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:52.154721 19224 raft_consensus.cc:1081] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Signalling peer 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:52.157248 19103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c"
dest_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:58088
I20260709 13:07:52.157864 19103 raft_consensus.cc:493] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:52.158254 19103 raft_consensus.cc:3060] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.165657 19103 raft_consensus.cc:515] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.167518 19103 leader_election.cc:290] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:52.168305 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:52.168601 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c055085a1d4959a878054bb8fd614c" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:52.168926 19028 raft_consensus.cc:3060] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.169219 19177 raft_consensus.cc:3055] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:52.169545 19177 raft_consensus.cc:740] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:07:52.170207 19177 consensus_queue.cc:260] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.171159 19177 raft_consensus.cc:3060] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.174355 19028 raft_consensus.cc:2468] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 2.
I20260709 13:07:52.175720 19067 leader_election.cc:304] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:52.176537 19177 raft_consensus.cc:2468] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 2.
I20260709 13:07:52.176690 19245 raft_consensus.cc:2804] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:52.177363 19245 raft_consensus.cc:697] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:52.178131 19245 consensus_queue.cc:237] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.185106 18923 catalog_manager.cc:5697] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 1 to 2, leader changed from 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67) to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.190830 19277 consensus_queue.cc:1048] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:52.197805 19249 raft_consensus.cc:993] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152: : Instructing follower 3f652e73eb8045f6bcdcfe70e5047960 to start an election
I20260709 13:07:52.198361 19223 raft_consensus.cc:1081] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Signalling peer 3f652e73eb8045f6bcdcfe70e5047960 to start an election
I20260709 13:07:52.200443 19176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39c48d1152f54851b1209458b609506e"
dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
 from {username='slave'} at 127.0.0.1:58850
I20260709 13:07:52.201033 19176 raft_consensus.cc:493] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:52.201349 19176 raft_consensus.cc:3060] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.205330 19249 consensus_queue.cc:1048] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:52.209359 19176 raft_consensus.cc:515] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.212265 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c48d1152f54851b1209458b609506e" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:52.212838 19028 raft_consensus.cc:3055] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:52.213162 19028 raft_consensus.cc:740] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:52.213842 19028 consensus_queue.cc:260] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.214542 19028 raft_consensus.cc:3060] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.220904 19028 raft_consensus.cc:2468] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 2.
I20260709 13:07:52.221143 19177 raft_consensus.cc:1240] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Rejecting Update request from peer 522cf294e90d4bf18c109548cda61152 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:52.222213 19141 leader_election.cc:304] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:52.223294 19224 raft_consensus.cc:2804] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:52.223690 19176 leader_election.cc:290] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 2 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:52.223888 19224 raft_consensus.cc:697] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 LEADER]: Becoming Leader. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:07:52.224881 19224 consensus_queue.cc:237] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.227097 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39c48d1152f54851b1209458b609506e" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:52.227706 19103 raft_consensus.cc:3060] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.234181 18924 catalog_manager.cc:5697] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 reported cstate change: term changed from 1 to 2, leader changed from 522cf294e90d4bf18c109548cda61152 (127.7.213.65) to 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.237442 19103 raft_consensus.cc:2468] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 2.
I20260709 13:07:52.338910 19228 raft_consensus.cc:993] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96: : Instructing follower 522cf294e90d4bf18c109548cda61152 to start an election
I20260709 13:07:52.339488 19245 raft_consensus.cc:1081] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Signalling peer 522cf294e90d4bf18c109548cda61152 to start an election
I20260709 13:07:52.341157 19028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "25af7d7302394721a33cd7017a84bea3"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
 from {username='slave'} at 127.0.0.1:51458
I20260709 13:07:52.341732 19028 raft_consensus.cc:493] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:52.342137 19028 raft_consensus.cc:3060] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.347332 19028 raft_consensus.cc:515] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.349092 19028 leader_election.cc:290] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:52.349858 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25af7d7302394721a33cd7017a84bea3" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:52.350450 19103 raft_consensus.cc:3055] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:52.350317 19176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25af7d7302394721a33cd7017a84bea3" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:52.350903 19103 raft_consensus.cc:740] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:52.351112 19176 raft_consensus.cc:3060] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.351711 19103 consensus_queue.cc:260] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.352738 19103 raft_consensus.cc:3060] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.357159 19176 raft_consensus.cc:2468] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 2.
I20260709 13:07:52.358402 18994 leader_election.cc:304] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:52.359179 19223 raft_consensus.cc:2804] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:52.359136 19103 raft_consensus.cc:2468] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 2.
I20260709 13:07:52.359951 19223 raft_consensus.cc:697] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:52.360829 19223 consensus_queue.cc:237] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.367604 18924 catalog_manager.cc:5697] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 1 to 2, leader changed from 47483c02b1e348278a40781fdc137f96 (127.7.213.66) to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.416905 19277 raft_consensus.cc:993] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152: : Instructing follower 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:52.417352 19223 raft_consensus.cc:1081] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Signalling peer 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:52.418825 19103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3016133635d74bcdbf0b50a9086778ab"
dest_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:58076
I20260709 13:07:52.419318 19103 raft_consensus.cc:493] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:52.419641 19103 raft_consensus.cc:3060] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.425376 19103 raft_consensus.cc:515] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.427191 19103 leader_election.cc:290] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:52.428001 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3016133635d74bcdbf0b50a9086778ab" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:52.428304 19176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3016133635d74bcdbf0b50a9086778ab" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:52.428625 19028 raft_consensus.cc:3055] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:52.428984 19176 raft_consensus.cc:3060] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.428988 19028 raft_consensus.cc:740] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:52.429972 19028 consensus_queue.cc:260] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.431275 19028 raft_consensus.cc:3060] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.432242 19177 raft_consensus.cc:1275] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.433852 19223 consensus_queue.cc:1048] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.437351 19176 raft_consensus.cc:2468] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 2.
I20260709 13:07:52.439515 19028 raft_consensus.cc:2468] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 2.
I20260709 13:07:52.439725 19069 leader_election.cc:304] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:52.441731 19245 raft_consensus.cc:2804] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:52.442543 19245 raft_consensus.cc:697] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:52.444000 19245 consensus_queue.cc:237] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:52.451768 19103 raft_consensus.cc:3060] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.453703 18923 catalog_manager.cc:5697] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 1 to 2, leader changed from 522cf294e90d4bf18c109548cda61152 (127.7.213.65) to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.460057 19103 raft_consensus.cc:1275] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.461624 19248 consensus_queue.cc:1048] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.578940 19028 raft_consensus.cc:1275] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.580330 19245 consensus_queue.cc:1048] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.588398 19176 raft_consensus.cc:3060] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:52.596884 19176 raft_consensus.cc:1275] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.598310 19228 consensus_queue.cc:1048] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.612049 19028 raft_consensus.cc:1275] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Refusing update from remote peer 3f652e73eb8045f6bcdcfe70e5047960: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.613572 19224 consensus_queue.cc:1048] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.622135 19103 raft_consensus.cc:1275] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Refusing update from remote peer 3f652e73eb8045f6bcdcfe70e5047960: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.623908 19291 consensus_queue.cc:1048] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.661180 19028 raft_consensus.cc:1275] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.662590 19228 consensus_queue.cc:1048] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.670679 19176 raft_consensus.cc:1275] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.672791 19245 consensus_queue.cc:1048] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.830134 19028 raft_consensus.cc:1275] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.831511 19245 consensus_queue.cc:1048] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.839433 19176 raft_consensus.cc:1275] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.840957 19228 consensus_queue.cc:1048] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.953651 19103 raft_consensus.cc:1275] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.955221 19248 consensus_queue.cc:1048] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:52.965247 19176 raft_consensus.cc:1275] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:52.966698 19223 consensus_queue.cc:1048] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:55.038707 18924 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44866:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:55.041349 18924 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:55.089713 19010 tablet_service.cc:1511] Processing CreateTablet for tablet e37171da97ae48e3a77e03d033b6d23c (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:55.091183 19010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e37171da97ae48e3a77e03d033b6d23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.090828 19093 tablet_service.cc:1511] Processing CreateTablet for tablet e37171da97ae48e3a77e03d033b6d23c (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:55.092351 19093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e37171da97ae48e3a77e03d033b6d23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.093648 19165 tablet_service.cc:1511] Processing CreateTablet for tablet e37171da97ae48e3a77e03d033b6d23c (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:55.094094 19016 tablet_service.cc:1511] Processing CreateTablet for tablet 57c432b444fb4596a3a38210d8b8936d (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:55.095276 19165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e37171da97ae48e3a77e03d033b6d23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.095468 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57c432b444fb4596a3a38210d8b8936d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.097978 19164 tablet_service.cc:1511] Processing CreateTablet for tablet 57c432b444fb4596a3a38210d8b8936d (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:55.099377 19164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57c432b444fb4596a3a38210d8b8936d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.102041 19161 tablet_service.cc:1511] Processing CreateTablet for tablet 7490273fa34949f08f5b3aa510be570b (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:55.103441 19161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7490273fa34949f08f5b3aa510be570b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.106572 19015 tablet_service.cc:1511] Processing CreateTablet for tablet 7490273fa34949f08f5b3aa510be570b (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:55.108922 19017 tablet_service.cc:1511] Processing CreateTablet for tablet c8983e4294f94da38f38ac2f6ea001fe (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:55.115522 19162 tablet_service.cc:1511] Processing CreateTablet for tablet c8983e4294f94da38f38ac2f6ea001fe (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:55.117803 19167 tablet_service.cc:1511] Processing CreateTablet for tablet 68989224c86646159ba1d1a5e722f160 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:55.111254 19012 tablet_service.cc:1511] Processing CreateTablet for tablet 68989224c86646159ba1d1a5e722f160 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:55.123412 19163 tablet_service.cc:1511] Processing CreateTablet for tablet 406df04366c743bc8a4b4542161f14f5 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:55.127107 19013 tablet_service.cc:1511] Processing CreateTablet for tablet 406df04366c743bc8a4b4542161f14f5 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:55.129001 19014 tablet_service.cc:1511] Processing CreateTablet for tablet 90a9ec28276348f787cd28248f7c55ef (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:55.129364 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 406df04366c743bc8a4b4542161f14f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.130357 19014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a9ec28276348f787cd28248f7c55ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.132477 19011 tablet_service.cc:1511] Processing CreateTablet for tablet 031f8922905b4356aab8310161abf4f6 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:55.133513 19017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8983e4294f94da38f38ac2f6ea001fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.127161 19018 tablet_service.cc:1511] Processing CreateTablet for tablet de4868f214f94bbfa2b0a721fd77d0f5 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:55.135018 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de4868f214f94bbfa2b0a721fd77d0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.136127 19167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68989224c86646159ba1d1a5e722f160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.136871 19319 tablet_bootstrap.cc:492] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:55.140098 19162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8983e4294f94da38f38ac2f6ea001fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.143906 19012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68989224c86646159ba1d1a5e722f160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.144661 19011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 031f8922905b4356aab8310161abf4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.143889 19319 tablet_bootstrap.cc:654] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.154515 19086 tablet_service.cc:1511] Processing CreateTablet for tablet 7490273fa34949f08f5b3aa510be570b (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:55.154887 19090 tablet_service.cc:1511] Processing CreateTablet for tablet c8983e4294f94da38f38ac2f6ea001fe (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:55.156225 19086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7490273fa34949f08f5b3aa510be570b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.157332 19091 tablet_service.cc:1511] Processing CreateTablet for tablet de4868f214f94bbfa2b0a721fd77d0f5 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:55.158708 19091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de4868f214f94bbfa2b0a721fd77d0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.161032 19088 tablet_service.cc:1511] Processing CreateTablet for tablet 68989224c86646159ba1d1a5e722f160 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:55.161885 19089 tablet_service.cc:1511] Processing CreateTablet for tablet 406df04366c743bc8a4b4542161f14f5 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:55.162551 19088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68989224c86646159ba1d1a5e722f160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.162238 19092 tablet_service.cc:1511] Processing CreateTablet for tablet 90a9ec28276348f787cd28248f7c55ef (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:55.163255 19089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 406df04366c743bc8a4b4542161f14f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.143576 19163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 406df04366c743bc8a4b4542161f14f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.146821 19015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7490273fa34949f08f5b3aa510be570b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.161041 19085 tablet_service.cc:1511] Processing CreateTablet for tablet 031f8922905b4356aab8310161abf4f6 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:55.164940 19085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 031f8922905b4356aab8310161abf4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.168118 19092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a9ec28276348f787cd28248f7c55ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.150295 19166 tablet_service.cc:1511] Processing CreateTablet for tablet de4868f214f94bbfa2b0a721fd77d0f5 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:55.172091 19090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8983e4294f94da38f38ac2f6ea001fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.172991 19166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de4868f214f94bbfa2b0a721fd77d0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.175704 19160 tablet_service.cc:1511] Processing CreateTablet for tablet 031f8922905b4356aab8310161abf4f6 (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:55.177179 19160 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 031f8922905b4356aab8310161abf4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.181350 19159 tablet_service.cc:1511] Processing CreateTablet for tablet 90a9ec28276348f787cd28248f7c55ef (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:55.182668 19159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a9ec28276348f787cd28248f7c55ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:55.216925 19093 tablet_service.cc:1511] Processing CreateTablet for tablet 57c432b444fb4596a3a38210d8b8936d (DEFAULT_TABLE table=soft_deleted_table [id=4cb475b15c304a899b25654133564002]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:55.228801 19093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57c432b444fb4596a3a38210d8b8936d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:55.261001 18974 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:55.275128 19320 tablet_bootstrap.cc:492] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:55.275519 19321 tablet_bootstrap.cc:492] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:55.286262 19320 tablet_bootstrap.cc:654] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.306396 19321 tablet_bootstrap.cc:654] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.311149 19319 tablet_bootstrap.cc:492] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:55.311664 19319 ts_tablet_manager.cc:1403] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.175s	user 0.011s	sys 0.004s
I20260709 13:07:55.315685 19319 raft_consensus.cc:359] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.316751 19319 raft_consensus.cc:385] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.317134 19319 raft_consensus.cc:740] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.318279 19319 consensus_queue.cc:260] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.319089 19321 tablet_bootstrap.cc:492] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:55.319772 19321 ts_tablet_manager.cc:1403] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.044s	user 0.006s	sys 0.011s
I20260709 13:07:55.323329 19321 raft_consensus.cc:359] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.324177 19321 raft_consensus.cc:385] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.324766 19321 raft_consensus.cc:740] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.326040 19319 ts_tablet_manager.cc:1434] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:07:55.325662 19321 consensus_queue.cc:260] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.327116 19319 tablet_bootstrap.cc:492] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:55.328516 19321 ts_tablet_manager.cc:1434] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:07:55.329806 19321 tablet_bootstrap.cc:492] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:55.332719 19320 tablet_bootstrap.cc:492] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:55.333385 19320 ts_tablet_manager.cc:1403] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.013s
I20260709 13:07:55.333810 19319 tablet_bootstrap.cc:654] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.336833 19320 raft_consensus.cc:359] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.337744 19320 raft_consensus.cc:385] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.338091 19320 raft_consensus.cc:740] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.338837 19321 tablet_bootstrap.cc:654] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.339036 19320 consensus_queue.cc:260] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.341856 19320 ts_tablet_manager.cc:1434] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:55.343592 19320 tablet_bootstrap.cc:492] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:55.353482 19320 tablet_bootstrap.cc:654] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.359105 19321 tablet_bootstrap.cc:492] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:55.359840 19321 ts_tablet_manager.cc:1403] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.014s
I20260709 13:07:55.364454 19321 raft_consensus.cc:359] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.365585 19321 raft_consensus.cc:385] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.365998 19321 raft_consensus.cc:740] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.367256 19321 consensus_queue.cc:260] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.370820 19321 ts_tablet_manager.cc:1434] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:07:55.372321 19321 tablet_bootstrap.cc:492] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:55.382773 19321 tablet_bootstrap.cc:654] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.397578 19321 tablet_bootstrap.cc:492] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:55.398196 19321 ts_tablet_manager.cc:1403] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.006s
I20260709 13:07:55.402830 19321 raft_consensus.cc:359] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.404016 19321 raft_consensus.cc:385] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.404441 19321 raft_consensus.cc:740] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.406098 19321 consensus_queue.cc:260] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.410292 19321 ts_tablet_manager.cc:1434] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.002s
I20260709 13:07:55.411832 19321 tablet_bootstrap.cc:492] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:55.421973 19321 tablet_bootstrap.cc:654] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.436338 19321 tablet_bootstrap.cc:492] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:55.437106 19321 ts_tablet_manager.cc:1403] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.011s
I20260709 13:07:55.444360 19321 raft_consensus.cc:359] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.445487 19321 raft_consensus.cc:385] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.445969 19321 raft_consensus.cc:740] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.446341 19307 raft_consensus.cc:493] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.447263 19321 consensus_queue.cc:260] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.447304 19307 raft_consensus.cc:515] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.451993 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37171da97ae48e3a77e03d033b6d23c" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:55.452837 19028 raft_consensus.cc:2468] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:55.454680 19141 leader_election.cc:304] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:55.457026 19330 raft_consensus.cc:2804] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.457660 19330 raft_consensus.cc:493] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.458145 19330 raft_consensus.cc:3060] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.465713 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37171da97ae48e3a77e03d033b6d23c" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:55.466089 19321 ts_tablet_manager.cc:1434] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.009s
I20260709 13:07:55.466991 19103 raft_consensus.cc:2468] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:55.467845 19321 tablet_bootstrap.cc:492] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:55.471977 19307 leader_election.cc:290] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:55.478735 19321 tablet_bootstrap.cc:654] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.486418 19330 raft_consensus.cc:515] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.491236 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37171da97ae48e3a77e03d033b6d23c" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:55.492035 19103 raft_consensus.cc:3060] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.492411 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37171da97ae48e3a77e03d033b6d23c" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:55.493259 19028 raft_consensus.cc:3060] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.504774 19330 leader_election.cc:290] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:55.507470 19103 raft_consensus.cc:2468] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 1.
I20260709 13:07:55.509563 19140 leader_election.cc:304] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:55.510866 19330 raft_consensus.cc:2804] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:55.511559 19330 raft_consensus.cc:697] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Becoming Leader. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:07:55.515698 19321 tablet_bootstrap.cc:492] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:55.516553 19321 ts_tablet_manager.cc:1403] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.006s
I20260709 13:07:55.517993 19307 raft_consensus.cc:493] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.518911 19307 raft_consensus.cc:515] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.512959 19330 consensus_queue.cc:237] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.523099 19321 raft_consensus.cc:359] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.524022 19321 raft_consensus.cc:385] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.524345 19321 raft_consensus.cc:740] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.525367 19321 consensus_queue.cc:260] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.528235 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
W20260709 13:07:55.529966 19140 leader_election.cc:343] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.536401 19321 ts_tablet_manager.cc:1434] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.003s
I20260709 13:07:55.537878 19321 tablet_bootstrap.cc:492] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:55.539595 19307 leader_election.cc:290] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:55.539705 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
W20260709 13:07:55.544325 19141 leader_election.cc:343] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.545071 19141 leader_election.cc:304] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960; no voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:55.548990 19331 raft_consensus.cc:2749] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:55.550935 19331 raft_consensus.cc:493] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.551970 19331 raft_consensus.cc:515] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.556046 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:55.565577 19321 tablet_bootstrap.cc:654] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:55.573521 19141 leader_election.cc:343] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.576094 19319 tablet_bootstrap.cc:492] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:55.576783 19319 ts_tablet_manager.cc:1403] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.250s	user 0.127s	sys 0.115s
I20260709 13:07:55.577131 19320 tablet_bootstrap.cc:492] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:55.577759 19320 ts_tablet_manager.cc:1403] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.234s	user 0.110s	sys 0.103s
I20260709 13:07:55.580205 19028 raft_consensus.cc:2468] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 1.
I20260709 13:07:55.580704 19319 raft_consensus.cc:359] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.581715 19319 raft_consensus.cc:385] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.582521 19319 raft_consensus.cc:740] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.583523 19331 leader_election.cc:290] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:55.583530 19319 consensus_queue.cc:260] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.585237 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:55.585273 19320 raft_consensus.cc:359] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.586360 19320 raft_consensus.cc:385] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.586922 19320 raft_consensus.cc:740] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.587510 19319 ts_tablet_manager.cc:1434] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:07:55.587929 19320 consensus_queue.cc:260] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
W20260709 13:07:55.589972 19140 leader_election.cc:343] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.590662 19140 leader_election.cc:304] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960; no voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:55.591557 19319 tablet_bootstrap.cc:492] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:55.590224 19320 ts_tablet_manager.cc:1434] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:07:55.593380 19320 tablet_bootstrap.cc:492] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:55.593523 19331 raft_consensus.cc:2749] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:55.610773 19321 tablet_bootstrap.cc:492] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:55.611588 19321 ts_tablet_manager.cc:1403] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.074s	user 0.019s	sys 0.005s
I20260709 13:07:55.613279 19320 tablet_bootstrap.cc:654] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.615837 19321 raft_consensus.cc:359] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.618305 19321 raft_consensus.cc:385] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.618760 19321 raft_consensus.cc:740] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.623041 19319 tablet_bootstrap.cc:654] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.625319 19320 tablet_bootstrap.cc:492] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:55.623591 19321 consensus_queue.cc:260] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.625895 19320 ts_tablet_manager.cc:1403] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.002s
I20260709 13:07:55.628366 19321 ts_tablet_manager.cc:1434] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 13:07:55.629841 19321 tablet_bootstrap.cc:492] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:55.629428 19320 raft_consensus.cc:359] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.630484 19320 raft_consensus.cc:385] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.630874 19320 raft_consensus.cc:740] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.631779 19320 consensus_queue.cc:260] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.634478 19320 ts_tablet_manager.cc:1434] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:07:55.635635 19320 tablet_bootstrap.cc:492] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:55.634478 18924 catalog_manager.cc:5697] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67). New cstate: current_term: 1 leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:55.644528 19320 tablet_bootstrap.cc:654] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.644527 19321 tablet_bootstrap.cc:654] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.660120 19319 tablet_bootstrap.cc:492] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:55.660768 19319 ts_tablet_manager.cc:1403] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.025s
I20260709 13:07:55.664635 19319 raft_consensus.cc:359] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.666247 19319 raft_consensus.cc:385] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.666635 19319 raft_consensus.cc:740] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.666712 19321 tablet_bootstrap.cc:492] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:55.667734 19319 consensus_queue.cc:260] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.667464 19321 ts_tablet_manager.cc:1403] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.009s
I20260709 13:07:55.670902 19320 tablet_bootstrap.cc:492] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:55.671568 19320 ts_tablet_manager.cc:1403] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.005s
I20260709 13:07:55.672035 19321 raft_consensus.cc:359] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.672837 19321 raft_consensus.cc:385] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.673177 19321 raft_consensus.cc:740] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.673950 19321 consensus_queue.cc:260] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.675191 19319 ts_tablet_manager.cc:1434] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:07:55.675336 19320 raft_consensus.cc:359] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.676229 19320 raft_consensus.cc:385] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.676550 19320 raft_consensus.cc:740] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.676915 19321 ts_tablet_manager.cc:1434] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:07:55.677523 19319 tablet_bootstrap.cc:492] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:55.677345 19320 consensus_queue.cc:260] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.681886 19320 ts_tablet_manager.cc:1434] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:07:55.682968 19320 tablet_bootstrap.cc:492] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:55.678526 19321 tablet_bootstrap.cc:492] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:55.685977 19319 tablet_bootstrap.cc:654] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.694126 19320 tablet_bootstrap.cc:654] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.697384 19321 tablet_bootstrap.cc:654] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.703055 19346 raft_consensus.cc:493] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.703689 19346 raft_consensus.cc:515] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.706082 19346 leader_election.cc:290] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:55.708400 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c432b444fb4596a3a38210d8b8936d" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
W20260709 13:07:55.710346 18991 leader_election.cc:343] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.711768 19320 tablet_bootstrap.cc:492] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:55.712393 19320 ts_tablet_manager.cc:1403] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.007s
I20260709 13:07:55.713619 19176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c432b444fb4596a3a38210d8b8936d" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:55.714394 19176 raft_consensus.cc:2468] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:55.716041 18994 leader_election.cc:304] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 47483c02b1e348278a40781fdc137f96
I20260709 13:07:55.716565 19320 raft_consensus.cc:359] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.717388 19320 raft_consensus.cc:385] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.717727 19320 raft_consensus.cc:740] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.718537 19320 consensus_queue.cc:260] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.719641 19223 raft_consensus.cc:2804] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.720069 19223 raft_consensus.cc:493] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.720445 19223 raft_consensus.cc:3060] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.721194 19320 ts_tablet_manager.cc:1434] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:55.725353 19321 tablet_bootstrap.cc:492] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:55.725940 19321 ts_tablet_manager.cc:1403] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.023s
I20260709 13:07:55.726609 19320 tablet_bootstrap.cc:492] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:55.729564 19321 raft_consensus.cc:359] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.730309 19321 raft_consensus.cc:385] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.730324 19319 tablet_bootstrap.cc:492] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:55.730661 19321 raft_consensus.cc:740] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.731264 19319 ts_tablet_manager.cc:1403] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.055s	user 0.022s	sys 0.027s
I20260709 13:07:55.731552 19321 consensus_queue.cc:260] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.734036 19321 ts_tablet_manager.cc:1434] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:07:55.734701 19320 tablet_bootstrap.cc:654] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.735219 19321 tablet_bootstrap.cc:492] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960: Bootstrap starting.
I20260709 13:07:55.735417 19319 raft_consensus.cc:359] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.736392 19319 raft_consensus.cc:385] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.736742 19319 raft_consensus.cc:740] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.737690 19319 consensus_queue.cc:260] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.742408 19321 tablet_bootstrap.cc:654] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.744243 19319 ts_tablet_manager.cc:1434] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:07:55.745796 19319 tablet_bootstrap.cc:492] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:55.755419 19319 tablet_bootstrap.cc:654] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.758702 19320 tablet_bootstrap.cc:492] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:55.759326 19320 ts_tablet_manager.cc:1403] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.015s
I20260709 13:07:55.759378 19321 tablet_bootstrap.cc:492] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960: No bootstrap required, opened a new log
I20260709 13:07:55.760135 19321 ts_tablet_manager.cc:1403] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.004s
I20260709 13:07:55.763013 19321 raft_consensus.cc:359] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.763778 19321 raft_consensus.cc:385] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.763083 19320 raft_consensus.cc:359] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.764091 19321 raft_consensus.cc:740] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.764286 19320 raft_consensus.cc:385] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.764940 19320 raft_consensus.cc:740] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.764876 19321 consensus_queue.cc:260] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.767323 19321 ts_tablet_manager.cc:1434] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:55.765988 19320 consensus_queue.cc:260] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.770316 19320 ts_tablet_manager.cc:1434] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:07:55.771396 19320 tablet_bootstrap.cc:492] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:55.772732 19319 tablet_bootstrap.cc:492] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:55.773300 19319 ts_tablet_manager.cc:1403] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.011s
I20260709 13:07:55.774995 19223 raft_consensus.cc:515] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.776531 19319 raft_consensus.cc:359] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.777302 19319 raft_consensus.cc:385] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.777603 19319 raft_consensus.cc:740] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.777725 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c432b444fb4596a3a38210d8b8936d" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:55.778689 19320 tablet_bootstrap.cc:654] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.778416 19319 consensus_queue.cc:260] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.779376 19228 raft_consensus.cc:493] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.780055 19228 raft_consensus.cc:515] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.782117 19228 leader_election.cc:290] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
W20260709 13:07:55.778975 18991 leader_election.cc:343] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.783478 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "031f8922905b4356aab8310161abf4f6" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:55.784198 19177 raft_consensus.cc:2468] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 0.
I20260709 13:07:55.785068 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "031f8922905b4356aab8310161abf4f6" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:55.785279 19069 leader_election.cc:304] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:55.785694 19028 raft_consensus.cc:2468] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 0.
I20260709 13:07:55.785728 19319 ts_tablet_manager.cc:1434] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:07:55.786254 19228 raft_consensus.cc:2804] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.788071 19228 raft_consensus.cc:493] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.788466 19228 raft_consensus.cc:3060] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.790315 19320 tablet_bootstrap.cc:492] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:55.790856 19320 ts_tablet_manager.cc:1403] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:07:55.793877 19320 raft_consensus.cc:359] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.794732 19320 raft_consensus.cc:385] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.795030 19320 raft_consensus.cc:740] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.795781 19320 consensus_queue.cc:260] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.796353 19228 raft_consensus.cc:515] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.787269 19319 tablet_bootstrap.cc:492] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:55.798166 19346 raft_consensus.cc:493] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.798411 19320 ts_tablet_manager.cc:1434] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:55.798729 19346 raft_consensus.cc:515] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.799515 19320 tablet_bootstrap.cc:492] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:55.803352 19176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68989224c86646159ba1d1a5e722f160" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:55.805920 19228 leader_election.cc:290] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:55.806368 19320 tablet_bootstrap.cc:654] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.806726 19319 tablet_bootstrap.cc:654] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.808764 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "031f8922905b4356aab8310161abf4f6" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:55.810429 19028 raft_consensus.cc:3060] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.809489 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "031f8922905b4356aab8310161abf4f6" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:55.814349 19177 raft_consensus.cc:3060] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.815526 19320 tablet_bootstrap.cc:492] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:55.816195 19320 ts_tablet_manager.cc:1403] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 13:07:55.818526 19228 raft_consensus.cc:493] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.819237 19228 raft_consensus.cc:515] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.820262 19320 raft_consensus.cc:359] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.821192 19320 raft_consensus.cc:385] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.821521 19320 raft_consensus.cc:740] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.822013 19228 leader_election.cc:290] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:55.822408 19320 consensus_queue.cc:260] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.824831 19175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:55.825565 19175 raft_consensus.cc:2468] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 0.
I20260709 13:07:55.826514 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:55.826841 19069 leader_election.cc:304] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:55.827714 19228 raft_consensus.cc:2804] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.828074 19320 ts_tablet_manager.cc:1434] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 13:07:55.828094 19228 raft_consensus.cc:493] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.828672 19228 raft_consensus.cc:3060] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:55.828588 19067 leader_election.cc:343] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.829176 19320 tablet_bootstrap.cc:492] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152: Bootstrap starting.
I20260709 13:07:55.836141 19320 tablet_bootstrap.cc:654] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.804216 19346 leader_election.cc:290] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:55.836467 19228 raft_consensus.cc:515] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.839881 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:55.840618 19228 leader_election.cc:290] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:55.839942 19175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:55.840370 19319 tablet_bootstrap.cc:492] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:55.841492 19175 raft_consensus.cc:3060] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.841696 19319 ts_tablet_manager.cc:1403] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.055s	user 0.007s	sys 0.021s
I20260709 13:07:55.846036 19319 raft_consensus.cc:359] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.846935 19319 raft_consensus.cc:385] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:55.847893 19319 raft_consensus.cc:740] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.849002 19319 consensus_queue.cc:260] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
W20260709 13:07:55.850695 19067 leader_election.cc:343] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:55.805019 19176 raft_consensus.cc:2468] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:55.852718 19319 ts_tablet_manager.cc:1434] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:07:55.853032 18994 leader_election.cc:304] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:55.854231 19319 tablet_bootstrap.cc:492] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:55.854400 19354 raft_consensus.cc:2804] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:55.855057 19354 raft_consensus.cc:493] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.855471 19354 raft_consensus.cc:3060] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.866065 19319 tablet_bootstrap.cc:654] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:55.873862 19176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c432b444fb4596a3a38210d8b8936d" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:55.875146 19176 raft_consensus.cc:3060] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:55.877004 19223 leader_election.cc:290] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:55.911327 19223 raft_consensus.cc:493] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:55.921516 19223 raft_consensus.cc:515] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.924618 19330 consensus_queue.cc:1048] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:07:55.947387 19354 raft_consensus.cc:515] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.961071 19319 tablet_bootstrap.cc:492] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:56.069077 19319 ts_tablet_manager.cc:1403] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.215s	user 0.056s	sys 0.067s
I20260709 13:07:55.981945 19330 raft_consensus.cc:493] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.071318 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68989224c86646159ba1d1a5e722f160" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:56.072027 19173 raft_consensus.cc:3060] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.071710 19330 raft_consensus.cc:515] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.074067 19330 leader_election.cc:290] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:55.965232 19174 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:56.075730 19174 raft_consensus.cc:2468] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:56.075906 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:56.076882 19102 raft_consensus.cc:2468] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:56.077039 18994 leader_election.cc:304] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:55.995438 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68989224c86646159ba1d1a5e722f160" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:56.013477 19100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:56.078650 19140 leader_election.cc:304] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:56.058444 19360 raft_consensus.cc:493] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.078927 19100 raft_consensus.cc:2468] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:56.080003 19360 raft_consensus.cc:515] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.025960 19351 raft_consensus.cc:493] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.080703 19173 raft_consensus.cc:2468] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:56.081444 19351 raft_consensus.cc:515] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.082239 18994 leader_election.cc:304] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:56.083854 19351 leader_election.cc:290] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.084626 19351 raft_consensus.cc:2804] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.073345 19319 raft_consensus.cc:359] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.085263 19319 raft_consensus.cc:385] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.085682 19319 raft_consensus.cc:740] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:55.960319 19177 raft_consensus.cc:2468] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 1.
I20260709 13:07:56.003268 19028 raft_consensus.cc:2468] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 1.
I20260709 13:07:56.019215 19228 raft_consensus.cc:493] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.090929 19069 leader_election.cc:304] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:56.087110 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68989224c86646159ba1d1a5e722f160" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:56.092177 19335 raft_consensus.cc:2804] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.093422 19335 raft_consensus.cc:697] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:56.094285 19335 consensus_queue.cc:237] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.096410 19100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7490273fa34949f08f5b3aa510be570b" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:56.097155 19100 raft_consensus.cc:2468] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:56.098374 19140 leader_election.cc:304] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:56.099267 19320 tablet_bootstrap.cc:492] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152: No bootstrap required, opened a new log
I20260709 13:07:56.099963 19320 ts_tablet_manager.cc:1403] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152: Time spent bootstrapping tablet: real 0.271s	user 0.093s	sys 0.164s
I20260709 13:07:56.100730 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7490273fa34949f08f5b3aa510be570b" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:56.101648 19102 raft_consensus.cc:2468] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:56.102212 19363 raft_consensus.cc:2804] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.102699 19363 raft_consensus.cc:493] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:55.990509 19223 leader_election.cc:290] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.087427 19360 leader_election.cc:290] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:56.038829 19346 raft_consensus.cc:493] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.102914 19320 raft_consensus.cc:359] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:55.962004 19175 raft_consensus.cc:2468] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 1.
I20260709 13:07:56.086789 19319 consensus_queue.cc:260] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.088932 19330 raft_consensus.cc:2804] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.103683 19320 raft_consensus.cc:385] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.010610 19307 consensus_queue.cc:1048] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.079s
I20260709 13:07:56.090890 19228 raft_consensus.cc:515] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.104243 19320 raft_consensus.cc:740] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.105908 18991 leader_election.cc:304] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:56.133633 19103 raft_consensus.cc:2468] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:56.132105 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7490273fa34949f08f5b3aa510be570b" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:56.104126 19223 raft_consensus.cc:2804] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.134370 19027 raft_consensus.cc:2468] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:56.134193 19320 consensus_queue.cc:260] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.134846 19223 raft_consensus.cc:697] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:56.135783 19366 raft_consensus.cc:493] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.135995 19364 raft_consensus.cc:2804] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.136420 19320 ts_tablet_manager.cc:1434] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152: Time spent starting tablet: real 0.036s	user 0.020s	sys 0.016s
I20260709 13:07:56.136381 19366 raft_consensus.cc:515] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.104045 19330 raft_consensus.cc:493] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.137821 19330 raft_consensus.cc:3060] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.137869 19364 raft_consensus.cc:493] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.138398 19364 raft_consensus.cc:3060] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.065116 19361 raft_consensus.cc:493] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.103826 19346 raft_consensus.cc:515] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.140265 19361 raft_consensus.cc:515] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.105530 19069 leader_election.cc:304] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 522cf294e90d4bf18c109548cda61152
I20260709 13:07:56.142742 19354 leader_election.cc:290] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.135690 19223 consensus_queue.cc:237] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
W20260709 13:07:56.144876 18994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:36609, user_credentials={real_user=slave}} blocked reactor thread for 62710.8us
I20260709 13:07:56.146665 19364 raft_consensus.cc:515] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.147167 19175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7490273fa34949f08f5b3aa510be570b" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:56.147405 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:56.148160 19027 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 0.
I20260709 13:07:56.149287 19067 leader_election.cc:304] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:56.150475 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:56.150943 19366 leader_election.cc:290] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.151245 19176 raft_consensus.cc:2468] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:56.151829 19366 raft_consensus.cc:2804] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.152441 19366 raft_consensus.cc:697] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:56.153549 19361 leader_election.cc:290] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.153283 19366 consensus_queue.cc:237] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.140257 19228 leader_election.cc:290] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.154281 19228 raft_consensus.cc:2804] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.154983 19228 raft_consensus.cc:493] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.155272 19228 raft_consensus.cc:3060] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.085045 19351 raft_consensus.cc:493] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.157816 19351 raft_consensus.cc:3060] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.158818 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:56.157781 19307 raft_consensus.cc:493] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.160801 19307 raft_consensus.cc:515] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.160985 18994 leader_election.cc:304] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 47483c02b1e348278a40781fdc137f96
I20260709 13:07:56.168457 19362 raft_consensus.cc:2804] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.169126 19362 raft_consensus.cc:697] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:56.170056 19362 consensus_queue.cc:237] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.169143 19176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:56.171586 19176 raft_consensus.cc:2468] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
I20260709 13:07:56.173308 19173 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 0.
W20260709 13:07:56.176390 19069 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.67:36609, user_credentials={real_user=slave}} blocked reactor thread for 70949.4us
I20260709 13:07:56.178433 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:56.179356 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:56.177212 18994 leader_election.cc:304] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:56.162868 19346 leader_election.cc:290] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.103046 19363 raft_consensus.cc:3060] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.179364 19173 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 0.
I20260709 13:07:56.188122 19100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:56.189145 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:56.189896 19026 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 0.
I20260709 13:07:56.192114 19100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7490273fa34949f08f5b3aa510be570b" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:56.192795 19100 raft_consensus.cc:3060] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.198735 19330 raft_consensus.cc:515] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.200542 18994 leader_election.cc:304] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:56.203434 18923 catalog_manager.cc:5697] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.205087 19365 raft_consensus.cc:2804] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.201630 19223 raft_consensus.cc:2804] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.202775 19141 leader_election.cc:304] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:56.205493 19365 raft_consensus.cc:493] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.205857 19223 raft_consensus.cc:493] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.212466 19223 raft_consensus.cc:3060] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:56.207027 18991 leader_election.cc:343] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:56.212571 19228 raft_consensus.cc:515] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.220436 19103 raft_consensus.cc:2393] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522cf294e90d4bf18c109548cda61152 in current term 1: Already voted for candidate 47483c02b1e348278a40781fdc137f96 in this term.
I20260709 13:07:56.225500 19319 ts_tablet_manager.cc:1434] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.156s	user 0.021s	sys 0.041s
I20260709 13:07:56.226634 19319 tablet_bootstrap.cc:492] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:56.226940 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:56.206346 19360 raft_consensus.cc:2804] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:56.228545 19360 raft_consensus.cc:493] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:56.228936 19360 raft_consensus.cc:3060] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.227262 18924 catalog_manager.cc:5697] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.218359 19365 raft_consensus.cc:3060] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.234731 19319 tablet_bootstrap.cc:654] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.219345 19307 leader_election.cc:290] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:56.239797 19365 raft_consensus.cc:515] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.241156 19351 raft_consensus.cc:515] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.242003 19365 leader_election.cc:290] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.242693 19364 leader_election.cc:290] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.205340 19363 raft_consensus.cc:515] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.243377 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:56.244464 19103 raft_consensus.cc:2393] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 522cf294e90d4bf18c109548cda61152 in current term 1: Already voted for candidate 47483c02b1e348278a40781fdc137f96 in this term.
I20260709 13:07:56.244504 19175 raft_consensus.cc:2393] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522cf294e90d4bf18c109548cda61152 in current term 1: Already voted for candidate 3f652e73eb8045f6bcdcfe70e5047960 in this term.
I20260709 13:07:56.245029 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68989224c86646159ba1d1a5e722f160" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:56.245882 19173 raft_consensus.cc:2393] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 47483c02b1e348278a40781fdc137f96 in current term 1: Already voted for candidate 522cf294e90d4bf18c109548cda61152 in this term.
W20260709 13:07:56.218920 19067 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.7.213.65:43671, user_credentials={real_user=slave}} blocked reactor thread for 64343.4us
I20260709 13:07:56.249993 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:56.250754 19103 raft_consensus.cc:2393] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f652e73eb8045f6bcdcfe70e5047960 in current term 1: Already voted for candidate 47483c02b1e348278a40781fdc137f96 in this term.
I20260709 13:07:56.254280 19176 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:56.256871 19100 raft_consensus.cc:2468] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:56.258066 18991 leader_election.cc:304] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:56.260717 19330 leader_election.cc:290] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:56.260126 19175 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7490273fa34949f08f5b3aa510be570b" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:56.261883 19175 raft_consensus.cc:2393] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 522cf294e90d4bf18c109548cda61152 in current term 1: Already voted for candidate 3f652e73eb8045f6bcdcfe70e5047960 in this term.
W20260709 13:07:56.264871 18974 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:56.265041 19365 raft_consensus.cc:2804] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.265353 19028 raft_consensus.cc:2393] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3f652e73eb8045f6bcdcfe70e5047960 in current term 1: Already voted for candidate 522cf294e90d4bf18c109548cda61152 in this term.
I20260709 13:07:56.265692 19365 raft_consensus.cc:697] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:56.267086 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68989224c86646159ba1d1a5e722f160" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:56.268659 19351 leader_election.cc:290] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.266611 19025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7490273fa34949f08f5b3aa510be570b" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:56.270545 19228 leader_election.cc:290] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.264952 19363 leader_election.cc:290] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:56.269194 19100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:56.274314 19100 raft_consensus.cc:3060] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.276285 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:56.275816 19223 raft_consensus.cc:515] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.285962 19103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7490273fa34949f08f5b3aa510be570b" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:56.287036 19103 raft_consensus.cc:2393] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f652e73eb8045f6bcdcfe70e5047960 in current term 1: Already voted for candidate 522cf294e90d4bf18c109548cda61152 in this term.
I20260709 13:07:56.282020 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:56.290764 19319 tablet_bootstrap.cc:492] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:56.291285 19319 ts_tablet_manager.cc:1403] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.065s	user 0.008s	sys 0.016s
I20260709 13:07:56.288655 19223 leader_election.cc:290] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.294112 19319 raft_consensus.cc:359] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.296006 19319 raft_consensus.cc:385] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.296351 19319 raft_consensus.cc:740] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.297837 19140 leader_election.cc:304] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960; no voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:56.298702 19363 raft_consensus.cc:2749] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.300496 19319 consensus_queue.cc:260] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.304379 19025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:56.304852 19067 leader_election.cc:304] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47483c02b1e348278a40781fdc137f96; no voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:56.305399 19025 raft_consensus.cc:2393] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 47483c02b1e348278a40781fdc137f96 in current term 1: Already voted for candidate 522cf294e90d4bf18c109548cda61152 in this term.
I20260709 13:07:56.305236 19177 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:56.306111 19177 raft_consensus.cc:2393] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 522cf294e90d4bf18c109548cda61152 in current term 1: Already voted for candidate 3f652e73eb8045f6bcdcfe70e5047960 in this term.
I20260709 13:07:56.306677 19366 raft_consensus.cc:3060] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.307564 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:56.308284 19102 raft_consensus.cc:3060] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.310477 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:56.311153 19173 raft_consensus.cc:3060] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.313042 19140 leader_election.cc:304] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960; no voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:56.313992 19363 raft_consensus.cc:2749] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.317828 19366 raft_consensus.cc:2749] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.296618 19360 raft_consensus.cc:515] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.321324 19067 leader_election.cc:304] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47483c02b1e348278a40781fdc137f96; no voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:56.322793 19176 raft_consensus.cc:2393] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 522cf294e90d4bf18c109548cda61152 in current term 1: Already voted for candidate 3f652e73eb8045f6bcdcfe70e5047960 in this term.
I20260709 13:07:56.286018 19365 consensus_queue.cc:237] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.325461 19366 raft_consensus.cc:2749] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.325803 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:56.327546 19102 raft_consensus.cc:2468] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:56.327168 19027 raft_consensus.cc:2393] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f652e73eb8045f6bcdcfe70e5047960 in current term 1: Already voted for candidate 522cf294e90d4bf18c109548cda61152 in this term.
I20260709 13:07:56.320758 19100 raft_consensus.cc:2468] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:56.330978 19319 ts_tablet_manager.cc:1434] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:07:56.332080 19319 tablet_bootstrap.cc:492] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96: Bootstrap starting.
I20260709 13:07:56.333915 18991 leader_election.cc:304] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:56.334359 19100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:56.335203 19100 raft_consensus.cc:2393] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3f652e73eb8045f6bcdcfe70e5047960 in current term 1: Already voted for candidate 47483c02b1e348278a40781fdc137f96 in this term.
I20260709 13:07:56.335064 18991 leader_election.cc:304] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 3f652e73eb8045f6bcdcfe70e5047960
I20260709 13:07:56.335901 19223 raft_consensus.cc:2804] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.336324 19223 raft_consensus.cc:697] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:56.330937 19360 leader_election.cc:290] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:56.337124 19223 consensus_queue.cc:237] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.338622 19365 raft_consensus.cc:2804] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:56.339135 19365 raft_consensus.cc:697] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:56.339926 19365 consensus_queue.cc:237] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.340282 18994 leader_election.cc:304] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522cf294e90d4bf18c109548cda61152; no voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96
I20260709 13:07:56.341149 19362 raft_consensus.cc:2749] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.342350 19173 raft_consensus.cc:2468] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:56.343149 19141 leader_election.cc:304] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960; no voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:56.344688 19360 raft_consensus.cc:2749] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.348611 19319 tablet_bootstrap.cc:654] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:56.354656 19319 tablet_bootstrap.cc:492] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96: No bootstrap required, opened a new log
I20260709 13:07:56.355129 19319 ts_tablet_manager.cc:1403] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 13:07:56.357951 19319 raft_consensus.cc:359] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.358590 19319 raft_consensus.cc:385] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:56.358865 19319 raft_consensus.cc:740] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Initialized, Role: FOLLOWER
I20260709 13:07:56.359468 19319 consensus_queue.cc:260] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.361462 19319 ts_tablet_manager.cc:1434] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:56.363673 18924 catalog_manager.cc:5697] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 1 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.372763 18923 catalog_manager.cc:5697] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.374289 18923 catalog_manager.cc:5697] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.375303 18923 catalog_manager.cc:5697] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.376489 19366 raft_consensus.cc:493] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.377020 19366 raft_consensus.cc:515] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.376832 18923 catalog_manager.cc:5697] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 1 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:56.379621 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c432b444fb4596a3a38210d8b8936d" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "522cf294e90d4bf18c109548cda61152" is_pre_election: true
I20260709 13:07:56.379913 19366 leader_election.cc:290] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.380110 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c432b444fb4596a3a38210d8b8936d" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:56.380944 19173 raft_consensus.cc:2393] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 47483c02b1e348278a40781fdc137f96 in current term 1: Already voted for candidate 522cf294e90d4bf18c109548cda61152 in this term.
I20260709 13:07:56.381810 19069 leader_election.cc:304] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 47483c02b1e348278a40781fdc137f96; no voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152
I20260709 13:07:56.382459 19366 raft_consensus.cc:3060] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.389362 19366 raft_consensus.cc:2749] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.402617 19365 raft_consensus.cc:493] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:56.403205 19365 raft_consensus.cc:515] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:56.405272 19365 leader_election.cc:290] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:56.405997 19100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:56.406328 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:56.407145 19173 raft_consensus.cc:2393] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522cf294e90d4bf18c109548cda61152 in current term 1: Already voted for candidate 47483c02b1e348278a40781fdc137f96 in this term.
I20260709 13:07:56.408084 18994 leader_election.cc:304] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522cf294e90d4bf18c109548cda61152; no voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96
I20260709 13:07:56.408746 19365 raft_consensus.cc:3060] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:56.414095 19365 raft_consensus.cc:2749] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:56.595304 19365 consensus_queue.cc:1048] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.606139 19351 consensus_queue.cc:1048] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.607407 19362 consensus_queue.cc:1048] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:56.621882 19362 consensus_queue.cc:1048] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.667166 19366 consensus_queue.cc:1048] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.680058 19366 consensus_queue.cc:1048] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.743120 19366 consensus_queue.cc:1048] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.753786 19228 consensus_queue.cc:1048] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.812705 19365 consensus_queue.cc:1048] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.828147 19223 consensus_queue.cc:1048] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.829754 19223 consensus_queue.cc:1048] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.835455 19223 consensus_queue.cc:1048] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.858625 19365 consensus_queue.cc:1048] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:56.870478 19365 consensus_queue.cc:1048] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:57.270310 18974 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:57.939065 19365 raft_consensus.cc:493] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:57.939682 19365 raft_consensus.cc:515] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:57.941623 19365 leader_election.cc:290] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:57.942621 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96" is_pre_election: true
I20260709 13:07:57.942658 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960" is_pre_election: true
I20260709 13:07:57.943352 19102 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:57.943519 19173 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 1.
I20260709 13:07:57.944641 18991 leader_election.cc:304] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:57.945356 19365 raft_consensus.cc:2804] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:57.945760 19365 raft_consensus.cc:493] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:57.946091 19365 raft_consensus.cc:3060] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:57.951524 19365 raft_consensus.cc:515] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:57.953238 19365 leader_election.cc:290] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:57.953882 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:57.954133 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:57.954463 19102 raft_consensus.cc:3060] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:57.954788 19173 raft_consensus.cc:3060] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:57.959553 19102 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 2.
I20260709 13:07:57.960219 19173 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 2.
I20260709 13:07:57.960827 18991 leader_election.cc:304] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:57.961756 19365 raft_consensus.cc:2804] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:57.962260 19365 raft_consensus.cc:697] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:57.963021 19365 consensus_queue.cc:237] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:57.971882 18926 catalog_manager.cc:5697] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 0 to 2, leader changed from <none> to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.541294 19102 raft_consensus.cc:1275] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:58.542858 19365 consensus_queue.cc:1048] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:58.545511 19173 raft_consensus.cc:1275] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:58.546860 19403 consensus_queue.cc:1048] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:58.636570 19176 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e37171da97ae48e3a77e03d033b6d23c"
dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
mode: GRACEFUL
new_leader_uuid: "522cf294e90d4bf18c109548cda61152"
 from {username='slave'} at 127.0.0.1:56680
I20260709 13:07:58.637147 19176 raft_consensus.cc:606] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Received request to transfer leadership to TS 522cf294e90d4bf18c109548cda61152
I20260709 13:07:58.648098 19102 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe"
dest_uuid: "47483c02b1e348278a40781fdc137f96"
mode: GRACEFUL
new_leader_uuid: "522cf294e90d4bf18c109548cda61152"
 from {username='slave'} at 127.0.0.1:36088
I20260709 13:07:58.648708 19102 raft_consensus.cc:606] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Received request to transfer leadership to TS 522cf294e90d4bf18c109548cda61152
I20260709 13:07:58.653786 19027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68989224c86646159ba1d1a5e722f160"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
mode: GRACEFUL
new_leader_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:07:58.654354 19027 raft_consensus.cc:606] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Received request to transfer leadership to TS 47483c02b1e348278a40781fdc137f96
I20260709 13:07:58.659910 19027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
mode: GRACEFUL
new_leader_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:07:58.660413 19027 raft_consensus.cc:606] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Received request to transfer leadership to TS 47483c02b1e348278a40781fdc137f96
I20260709 13:07:58.665751 19027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
mode: GRACEFUL
new_leader_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:07:58.666340 19027 raft_consensus.cc:606] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Received request to transfer leadership to TS 47483c02b1e348278a40781fdc137f96
I20260709 13:07:58.674291 19027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
mode: GRACEFUL
new_leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
 from {username='slave'} at 127.0.0.1:54420
I20260709 13:07:58.674778 19027 raft_consensus.cc:606] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Received request to transfer leadership to TS 3f652e73eb8045f6bcdcfe70e5047960
I20260709 13:07:58.733915 19403 raft_consensus.cc:993] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152: : Instructing follower 3f652e73eb8045f6bcdcfe70e5047960 to start an election
I20260709 13:07:58.734404 19415 raft_consensus.cc:1081] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Signalling peer 3f652e73eb8045f6bcdcfe70e5047960 to start an election
I20260709 13:07:58.736011 19176 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef"
dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
 from {username='slave'} at 127.0.0.1:58850
I20260709 13:07:58.736521 19176 raft_consensus.cc:493] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.736832 19176 raft_consensus.cc:3060] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.741381 19176 raft_consensus.cc:515] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:58.742867 19176 leader_election.cc:290] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 2 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:07:58.743985 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:58.744035 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a9ec28276348f787cd28248f7c55ef" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:58.744573 19027 raft_consensus.cc:3055] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.745016 19027 raft_consensus.cc:740] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:58.745043 19102 raft_consensus.cc:3060] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.745792 19027 consensus_queue.cc:260] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:58.746762 19027 raft_consensus.cc:3060] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.750636 19102 raft_consensus.cc:2468] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 2.
I20260709 13:07:58.752153 19140 leader_election.cc:304] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:58.752178 19027 raft_consensus.cc:2468] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 2.
I20260709 13:07:58.753119 19410 raft_consensus.cc:2804] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.753609 19410 raft_consensus.cc:697] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 LEADER]: Becoming Leader. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:07:58.754439 19410 consensus_queue.cc:237] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:58.764249 18924 catalog_manager.cc:5697] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 reported cstate change: term changed from 1 to 2, leader changed from 522cf294e90d4bf18c109548cda61152 (127.7.213.65) to 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:58.767455 19396 raft_consensus.cc:993] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960: : Instructing follower 522cf294e90d4bf18c109548cda61152 to start an election
I20260709 13:07:58.768051 19410 raft_consensus.cc:1081] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Signalling peer 522cf294e90d4bf18c109548cda61152 to start an election
I20260709 13:07:58.769538 19027 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e37171da97ae48e3a77e03d033b6d23c"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
 from {username='slave'} at 127.0.0.1:51452
I20260709 13:07:58.770109 19027 raft_consensus.cc:493] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.770490 19027 raft_consensus.cc:3060] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.777707 19027 raft_consensus.cc:515] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:58.780421 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37171da97ae48e3a77e03d033b6d23c" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:58.780932 19027 leader_election.cc:290] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:58.780982 19102 raft_consensus.cc:3060] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.782255 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e37171da97ae48e3a77e03d033b6d23c" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:58.782898 19173 raft_consensus.cc:3055] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.783211 19173 raft_consensus.cc:740] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:07:58.784125 19173 consensus_queue.cc:260] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:58.785306 19173 raft_consensus.cc:3060] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.788432 19102 raft_consensus.cc:2468] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 2.
I20260709 13:07:58.789661 18991 leader_election.cc:304] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47483c02b1e348278a40781fdc137f96, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:58.790369 19403 raft_consensus.cc:2804] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.790850 19403 raft_consensus.cc:697] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:58.791631 19403 consensus_queue.cc:237] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:58.792172 19173 raft_consensus.cc:2468] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 2.
I20260709 13:07:58.799880 18924 catalog_manager.cc:5697] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 1 to 2, leader changed from 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67) to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.950701 19415 raft_consensus.cc:993] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152: : Instructing follower 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:58.951170 19403 raft_consensus.cc:1081] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Signalling peer 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:58.952710 19102 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5"
dest_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:58076
I20260709 13:07:58.953153 19102 raft_consensus.cc:493] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.953389 19102 raft_consensus.cc:3060] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.957803 19102 raft_consensus.cc:515] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:58.959475 19102 leader_election.cc:290] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:58.960274 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:58.960574 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de4868f214f94bbfa2b0a721fd77d0f5" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:58.960928 19027 raft_consensus.cc:3055] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:58.961177 19173 raft_consensus.cc:3060] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.961337 19027 raft_consensus.cc:740] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:58.962190 19027 consensus_queue.cc:260] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:58.963112 19027 raft_consensus.cc:3060] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:58.966408 19173 raft_consensus.cc:2468] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 2.
I20260709 13:07:58.967680 19069 leader_election.cc:304] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:58.968415 19027 raft_consensus.cc:2468] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 2.
I20260709 13:07:58.968533 19366 raft_consensus.cc:2804] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:58.969901 19366 raft_consensus.cc:697] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:58.970759 19366 consensus_queue.cc:237] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:58.978636 18924 catalog_manager.cc:5697] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 1 to 2, leader changed from 522cf294e90d4bf18c109548cda61152 (127.7.213.65) to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:58.992532 19415 raft_consensus.cc:993] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152: : Instructing follower 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:58.992934 19403 raft_consensus.cc:1081] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Signalling peer 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:58.994256 19102 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68989224c86646159ba1d1a5e722f160"
dest_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:58076
I20260709 13:07:58.994771 19102 raft_consensus.cc:493] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:58.995081 19102 raft_consensus.cc:3060] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:59.000882 19102 raft_consensus.cc:515] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:59.002441 19102 leader_election.cc:290] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:59.003320 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68989224c86646159ba1d1a5e722f160" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:59.003724 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68989224c86646159ba1d1a5e722f160" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:59.004136 19027 raft_consensus.cc:3055] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:59.004521 19027 raft_consensus.cc:740] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:59.004530 19173 raft_consensus.cc:3060] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:59.005708 19027 consensus_queue.cc:260] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:59.007140 19027 raft_consensus.cc:3060] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:59.013150 19173 raft_consensus.cc:2468] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 2.
I20260709 13:07:59.014544 19069 leader_election.cc:304] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:59.015023 19027 raft_consensus.cc:2468] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 2.
I20260709 13:07:59.015590 19366 raft_consensus.cc:2804] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:59.016218 19366 raft_consensus.cc:697] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:59.017027 19366 consensus_queue.cc:237] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:59.024356 19422 raft_consensus.cc:993] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96: : Instructing follower 522cf294e90d4bf18c109548cda61152 to start an election
I20260709 13:07:59.024858 19366 raft_consensus.cc:1081] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Signalling peer 522cf294e90d4bf18c109548cda61152 to start an election
I20260709 13:07:59.025916 18924 catalog_manager.cc:5697] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 1 to 2, leader changed from 522cf294e90d4bf18c109548cda61152 (127.7.213.65) to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 2 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:59.027319 19027 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
 from {username='slave'} at 127.0.0.1:51458
I20260709 13:07:59.027904 19027 raft_consensus.cc:493] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:59.028263 19027 raft_consensus.cc:3060] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:59.035394 19027 raft_consensus.cc:515] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:59.037367 19027 leader_election.cc:290] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Requested vote from peers 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:59.038076 19102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:07:59.038312 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8983e4294f94da38f38ac2f6ea001fe" candidate_uuid: "522cf294e90d4bf18c109548cda61152" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:59.038789 19102 raft_consensus.cc:3055] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:59.038968 19173 raft_consensus.cc:3060] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:59.039136 19102 raft_consensus.cc:740] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:59.039954 19102 consensus_queue.cc:260] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:59.041076 19102 raft_consensus.cc:3060] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:59.046026 19173 raft_consensus.cc:2468] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 2.
I20260709 13:07:59.047255 18994 leader_election.cc:304] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:07:59.047612 19102 raft_consensus.cc:2468] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522cf294e90d4bf18c109548cda61152 in term 2.
I20260709 13:07:59.048136 19403 raft_consensus.cc:2804] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:59.048681 19403 raft_consensus.cc:697] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Becoming Leader. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:59.049511 19403 consensus_queue.cc:237] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:59.058234 18924 catalog_manager.cc:5697] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 reported cstate change: term changed from 1 to 2, leader changed from 47483c02b1e348278a40781fdc137f96 (127.7.213.66) to 522cf294e90d4bf18c109548cda61152 (127.7.213.65). New cstate: current_term: 2 leader_uuid: "522cf294e90d4bf18c109548cda61152" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:59.170514 19415 raft_consensus.cc:993] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152: : Instructing follower 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:59.171038 19403 raft_consensus.cc:1081] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Signalling peer 47483c02b1e348278a40781fdc137f96 to start an election
I20260709 13:07:59.172705 19102 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5"
dest_uuid: "47483c02b1e348278a40781fdc137f96"
 from {username='slave'} at 127.0.0.1:58076
I20260709 13:07:59.173189 19102 raft_consensus.cc:493] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:59.173452 19102 raft_consensus.cc:3060] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:59.178231 19102 raft_consensus.cc:515] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:59.180433 19102 leader_election.cc:290] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 3 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67:36609)
I20260709 13:07:59.181291 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:07:59.182015 19027 raft_consensus.cc:3055] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:59.182070 19173 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "406df04366c743bc8a4b4542161f14f5" candidate_uuid: "47483c02b1e348278a40781fdc137f96" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
I20260709 13:07:59.182360 19027 raft_consensus.cc:740] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:07:59.182655 19173 raft_consensus.cc:3060] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:59.183240 19027 consensus_queue.cc:260] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:59.184331 19027 raft_consensus.cc:3060] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:59.187623 19173 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 3.
I20260709 13:07:59.188789 19069 leader_election.cc:304] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 47483c02b1e348278a40781fdc137f96; no voters: 
I20260709 13:07:59.189657 19366 raft_consensus.cc:2804] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:59.190081 19366 raft_consensus.cc:697] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 3 LEADER]: Becoming Leader. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:07:59.190598 19027 raft_consensus.cc:2468] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47483c02b1e348278a40781fdc137f96 in term 3.
I20260709 13:07:59.190802 19366 consensus_queue.cc:237] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:07:59.197953 18924 catalog_manager.cc:5697] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 reported cstate change: term changed from 2 to 3, leader changed from 522cf294e90d4bf18c109548cda61152 (127.7.213.65) to 47483c02b1e348278a40781fdc137f96 (127.7.213.66). New cstate: current_term: 3 leader_uuid: "47483c02b1e348278a40781fdc137f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:59.208705 19102 raft_consensus.cc:1275] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.210043 19426 consensus_queue.cc:1048] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.221014 19173 raft_consensus.cc:1275] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.222182 19415 consensus_queue.cc:1048] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.267432 19102 raft_consensus.cc:1275] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Refusing update from remote peer 3f652e73eb8045f6bcdcfe70e5047960: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.268810 19410 consensus_queue.cc:1048] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.279561 19027 raft_consensus.cc:1275] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Refusing update from remote peer 3f652e73eb8045f6bcdcfe70e5047960: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.282188 19433 consensus_queue.cc:1048] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.360040 19173 raft_consensus.cc:1275] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.361367 19366 consensus_queue.cc:1048] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.370456 19027 raft_consensus.cc:1275] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.375197 19438 consensus_queue.cc:1048] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:59.438484 19173 raft_consensus.cc:1275] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.439937 19415 consensus_queue.cc:1048] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.449779 19102 raft_consensus.cc:1275] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Refusing update from remote peer 522cf294e90d4bf18c109548cda61152: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.452383 19403 consensus_queue.cc:1048] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:59.556901 19173 raft_consensus.cc:1275] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.558205 19437 consensus_queue.cc:1048] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.567004 19027 raft_consensus.cc:1275] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:59.568868 19422 consensus_queue.cc:1048] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.695669 19173 raft_consensus.cc:1275] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 3 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:59.696946 19422 consensus_queue.cc:1048] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:59.702492 19027 raft_consensus.cc:1275] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 3 FOLLOWER]: Refusing update from remote peer 47483c02b1e348278a40781fdc137f96: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:07:59.704274 19422 consensus_queue.cc:1048] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:01.678872 18926 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44866:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:08:01.703478 18926 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44866:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:08:01.730489  8021 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:08:02.014300 19480 log.cc:927] Time spent T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:08:02.013396 19493 tablet.cc:1724] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.022181 19497 tablet.cc:1724] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.023641 19023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11"
dest_uuid: "522cf294e90d4bf18c109548cda61152"
mode: GRACEFUL
new_leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
 from {username='slave'} at 127.0.0.1:35138
I20260709 13:08:02.024783 19023 raft_consensus.cc:606] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Received request to transfer leadership to TS 3f652e73eb8045f6bcdcfe70e5047960
I20260709 13:08:02.108644 19504 tablet.cc:1724] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:02.117028 19498 log.cc:927] Time spent T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
I20260709 13:08:02.120793 19507 tablet.cc:1724] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.156576 19509 tablet.cc:1724] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.162129 19511 tablet.cc:1724] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:02.171389 19494 log.cc:927] Time spent T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960: Append to log took a long time: real 0.144s	user 0.003s	sys 0.000s
I20260709 13:08:02.187252 19473 raft_consensus.cc:993] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152: : Instructing follower 3f652e73eb8045f6bcdcfe70e5047960 to start an election
I20260709 13:08:02.187861 19473 raft_consensus.cc:1081] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Signalling peer 3f652e73eb8045f6bcdcfe70e5047960 to start an election
I20260709 13:08:02.190342 19175 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11"
dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
 from {username='slave'} at 127.0.0.1:58850
I20260709 13:08:02.191212 19175 raft_consensus.cc:493] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:02.191716 19175 raft_consensus.cc:3060] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.225039 19510 tablet.cc:1724] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.231248 19516 tablet.cc:1724] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.234931 19507 tablet.cc:1724] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.263818 19515 tablet.cc:1724] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.269146 19519 tablet.cc:1724] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.272048 19511 tablet.cc:1724] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.277899 19518 tablet.cc:1724] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.296653 19515 tablet.cc:1724] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.302933 19504 tablet.cc:1724] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:08:02.307281 19069 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.7.213.67:36609, user_credentials={real_user=slave}} blocked reactor thread for 68517.8us
I20260709 13:08:02.322754 19507 tablet.cc:1724] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.315023 19175 raft_consensus.cc:515] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:08:02.331547 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:08:02.332350 19024 raft_consensus.cc:3055] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.332693 19024 raft_consensus.cc:740] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 522cf294e90d4bf18c109548cda61152, State: Running, Role: LEADER
I20260709 13:08:02.333232 19098 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7"
dest_uuid: "47483c02b1e348278a40781fdc137f96"
mode: GRACEFUL
new_leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
 from {username='slave'} at 127.0.0.1:44984
I20260709 13:08:02.333580 19100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4400e1db38f4227bc7ddcde78d93a11" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:08:02.333938 19098 raft_consensus.cc:606] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Received request to transfer leadership to TS 3f652e73eb8045f6bcdcfe70e5047960
I20260709 13:08:02.334264 19100 raft_consensus.cc:3060] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.334831 19024 consensus_queue.cc:260] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:08:02.336846  8021 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:02.339917  8021 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:02.341837 19024 raft_consensus.cc:3060] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.343639 19497 tablet.cc:1724] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.334904 19493 tablet.cc:1724] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.350925 19024 raft_consensus.cc:2468] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 2.
I20260709 13:08:02.341881 19513 tablet.cc:1724] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.347769 19516 tablet.cc:1724] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.359441 19510 tablet.cc:1724] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.340945 19175 leader_election.cc:290] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 2 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:08:02.366648 19141 leader_election.cc:304] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:08:02.367919 19492 raft_consensus.cc:2804] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:02.368393 19492 raft_consensus.cc:697] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 LEADER]: Becoming Leader. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:08:02.369438 19492 consensus_queue.cc:237] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } }
I20260709 13:08:02.361363 19509 tablet.cc:1724] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.396091 19519 tablet.cc:1724] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.359505 19511 tablet.cc:1724] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.404695 19515 tablet.cc:1724] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.418231 19100 raft_consensus.cc:2468] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 2.
I20260709 13:08:02.424293 18923 catalog_manager.cc:5697] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 reported cstate change: term changed from 1 to 2, leader changed from 522cf294e90d4bf18c109548cda61152 (127.7.213.65) to 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:02.419044 19515 tablet.cc:1724] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.433364 19513 tablet.cc:1724] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:08:02.458967 18926 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121: soft_deleted_table [id=4cb475b15c304a899b25654133564002] alter complete (version 1)
I20260709 13:08:02.476548 19521 raft_consensus.cc:993] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96: : Instructing follower 3f652e73eb8045f6bcdcfe70e5047960 to start an election
I20260709 13:08:02.477200 19520 raft_consensus.cc:1081] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Signalling peer 3f652e73eb8045f6bcdcfe70e5047960 to start an election
I20260709 13:08:02.478852 19173 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7"
dest_uuid: "3f652e73eb8045f6bcdcfe70e5047960"
 from {username='slave'} at 127.0.0.1:58864
I20260709 13:08:02.479363 19173 raft_consensus.cc:493] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:02.479683 19173 raft_consensus.cc:3060] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.484687 19173 raft_consensus.cc:515] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:08:02.485975 19173 leader_election.cc:290] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 2 election: Requested vote from peers 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671), 47483c02b1e348278a40781fdc137f96 (127.7.213.66:40053)
I20260709 13:08:02.486816 19024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "522cf294e90d4bf18c109548cda61152"
I20260709 13:08:02.487068 19100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed00071d9a064800985eeae64cdda3e7" candidate_uuid: "3f652e73eb8045f6bcdcfe70e5047960" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "47483c02b1e348278a40781fdc137f96"
I20260709 13:08:02.487531 19024 raft_consensus.cc:3060] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.487674 19100 raft_consensus.cc:3055] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:02.488101 19100 raft_consensus.cc:740] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 47483c02b1e348278a40781fdc137f96, State: Running, Role: LEADER
I20260709 13:08:02.488952 19100 consensus_queue.cc:260] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:08:02.490274 19100 raft_consensus.cc:3060] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:02.494762 19024 raft_consensus.cc:2468] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 2.
I20260709 13:08:02.496237 19141 leader_election.cc:304] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f652e73eb8045f6bcdcfe70e5047960, 522cf294e90d4bf18c109548cda61152; no voters: 
I20260709 13:08:02.497028 19492 raft_consensus.cc:2804] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:02.497278 19100 raft_consensus.cc:2468] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f652e73eb8045f6bcdcfe70e5047960 in term 2.
I20260709 13:08:02.497483 19492 raft_consensus.cc:697] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 LEADER]: Becoming Leader. State: Replica: 3f652e73eb8045f6bcdcfe70e5047960, State: Running, Role: LEADER
I20260709 13:08:02.498725 19492 consensus_queue.cc:237] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } }
I20260709 13:08:02.506827 18926 catalog_manager.cc:5697] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 reported cstate change: term changed from 1 to 2, leader changed from 47483c02b1e348278a40781fdc137f96 (127.7.213.66) to 3f652e73eb8045f6bcdcfe70e5047960 (127.7.213.67). New cstate: current_term: 2 leader_uuid: "3f652e73eb8045f6bcdcfe70e5047960" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f652e73eb8045f6bcdcfe70e5047960" member_type: VOTER last_known_addr { host: "127.7.213.67" port: 36609 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:02.760334 19100 raft_consensus.cc:1275] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Refusing update from remote peer 3f652e73eb8045f6bcdcfe70e5047960: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.761435 19492 consensus_queue.cc:1048] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:02.771312 19024 raft_consensus.cc:1275] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Refusing update from remote peer 3f652e73eb8045f6bcdcfe70e5047960: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.773043 19449 consensus_queue.cc:1048] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:02.940819 19100 raft_consensus.cc:1275] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Refusing update from remote peer 3f652e73eb8045f6bcdcfe70e5047960: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.942684 19449 consensus_queue.cc:1048] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47483c02b1e348278a40781fdc137f96" member_type: VOTER last_known_addr { host: "127.7.213.66" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:02.951854 19024 raft_consensus.cc:1275] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Refusing update from remote peer 3f652e73eb8045f6bcdcfe70e5047960: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:02.954454 19530 consensus_queue.cc:1048] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522cf294e90d4bf18c109548cda61152" member_type: VOTER last_known_addr { host: "127.7.213.65" port: 43671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.402772  8021 tablet_server.cc:179] TabletServer@127.7.213.65:0 shutting down...
W20260709 13:08:03.552816 19141 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 13:08:03.562189 19141 consensus_peers.cc:597] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 -> Peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Couldn't send request to peer 522cf294e90d4bf18c109548cda61152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:03.565419 19067 consensus_peers.cc:597] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 -> Peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Couldn't send request to peer 522cf294e90d4bf18c109548cda61152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:03.566211 19067 consensus_peers.cc:597] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 -> Peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Couldn't send request to peer 522cf294e90d4bf18c109548cda61152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:03.618898 19141 consensus_peers.cc:597] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 -> Peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Couldn't send request to peer 522cf294e90d4bf18c109548cda61152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.627068  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:03.628036  8021 tablet_replica.cc:333] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.628803  8021 raft_consensus.cc:2243] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.629642  8021 raft_consensus.cc:2272] T c8983e4294f94da38f38ac2f6ea001fe P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.631924  8021 tablet_replica.cc:333] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.632612  8021 raft_consensus.cc:2243] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.633111  8021 raft_consensus.cc:2272] T 90a9ec28276348f787cd28248f7c55ef P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.635154  8021 tablet_replica.cc:333] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.635833  8021 raft_consensus.cc:2243] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.636605  8021 raft_consensus.cc:2272] T 7490273fa34949f08f5b3aa510be570b P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.638743  8021 tablet_replica.cc:333] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.639345  8021 raft_consensus.cc:2243] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.640205  8021 raft_consensus.cc:2272] T 57c432b444fb4596a3a38210d8b8936d P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.642202  8021 tablet_replica.cc:333] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.642767  8021 raft_consensus.cc:2243] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.643530  8021 raft_consensus.cc:2272] T e37171da97ae48e3a77e03d033b6d23c P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.645601  8021 tablet_replica.cc:333] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.646221  8021 raft_consensus.cc:2243] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.646749  8021 raft_consensus.cc:2272] T 39c48d1152f54851b1209458b609506e P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.648993  8021 tablet_replica.cc:333] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.649516  8021 raft_consensus.cc:2243] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.650321  8021 raft_consensus.cc:2272] T 1f35f40f03074dd4862bc0e80fd8caed P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.652472  8021 tablet_replica.cc:333] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.653038  8021 raft_consensus.cc:2243] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.653916  8021 raft_consensus.cc:2272] T a20d75390629460099ed0728baefb3ab P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.656097  8021 tablet_replica.cc:333] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.656684  8021 raft_consensus.cc:2243] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.657176  8021 raft_consensus.cc:2272] T 68989224c86646159ba1d1a5e722f160 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.659109  8021 tablet_replica.cc:333] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.659767  8021 raft_consensus.cc:2243] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.660266  8021 raft_consensus.cc:2272] T 031f8922905b4356aab8310161abf4f6 P 522cf294e90d4bf18c109548cda61152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.662519  8021 tablet_replica.cc:333] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.663110  8021 raft_consensus.cc:2243] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.663573  8021 raft_consensus.cc:2272] T ed00071d9a064800985eeae64cdda3e7 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.666086  8021 tablet_replica.cc:333] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.666694  8021 raft_consensus.cc:2243] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.667315  8021 raft_consensus.cc:2272] T 406df04366c743bc8a4b4542161f14f5 P 522cf294e90d4bf18c109548cda61152 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.670034  8021 tablet_replica.cc:333] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.670923  8021 raft_consensus.cc:2243] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:08:03.671795 19141 consensus_peers.cc:597] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 -> Peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Couldn't send request to peer 522cf294e90d4bf18c109548cda61152. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.672042  8021 raft_consensus.cc:2272] T 25af7d7302394721a33cd7017a84bea3 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.674360  8021 tablet_replica.cc:333] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.674872  8021 raft_consensus.cc:2243] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.675328  8021 raft_consensus.cc:2272] T 3016133635d74bcdbf0b50a9086778ab P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.677345  8021 tablet_replica.cc:333] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.677913  8021 raft_consensus.cc:2243] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.678360  8021 raft_consensus.cc:2272] T 0864698179204de6a0c3ab6f41959297 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.680672  8021 tablet_replica.cc:333] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.681152  8021 raft_consensus.cc:2243] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.681578  8021 raft_consensus.cc:2272] T de4868f214f94bbfa2b0a721fd77d0f5 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.683447  8021 tablet_replica.cc:333] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.684020  8021 raft_consensus.cc:2243] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.684424  8021 raft_consensus.cc:2272] T c4400e1db38f4227bc7ddcde78d93a11 P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.687318  8021 tablet_replica.cc:333] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152: stopping tablet replica
I20260709 13:08:03.687918  8021 raft_consensus.cc:2243] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.688373  8021 raft_consensus.cc:2272] T c4c055085a1d4959a878054bb8fd614c P 522cf294e90d4bf18c109548cda61152 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.733616  8021 tablet_server.cc:196] TabletServer@127.7.213.65:0 shutdown complete.
W20260709 13:08:03.763221 19141 consensus_peers.cc:597] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 -> Peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Couldn't send request to peer 522cf294e90d4bf18c109548cda61152. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:43671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.775063  8021 tablet_server.cc:179] TabletServer@127.7.213.66:0 shutting down...
W20260709 13:08:03.794890 19067 consensus_peers.cc:597] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 -> Peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Couldn't send request to peer 522cf294e90d4bf18c109548cda61152. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:43671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:03.810237 19067 consensus_peers.cc:597] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 -> Peer 522cf294e90d4bf18c109548cda61152 (127.7.213.65:43671): Couldn't send request to peer 522cf294e90d4bf18c109548cda61152. Status: Network error: Client connection negotiation failed: client connection to 127.7.213.65:43671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.810832  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:03.811641  8021 tablet_replica.cc:333] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.812633  8021 raft_consensus.cc:2243] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.813508  8021 raft_consensus.cc:2272] T 406df04366c743bc8a4b4542161f14f5 P 47483c02b1e348278a40781fdc137f96 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.815529  8021 tablet_replica.cc:333] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.816156  8021 raft_consensus.cc:2243] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.816609  8021 raft_consensus.cc:2272] T 90a9ec28276348f787cd28248f7c55ef P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.818514  8021 tablet_replica.cc:333] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.819061  8021 raft_consensus.cc:2243] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.819494  8021 raft_consensus.cc:2272] T 7490273fa34949f08f5b3aa510be570b P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.821419  8021 tablet_replica.cc:333] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.822000  8021 raft_consensus.cc:2243] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.822438  8021 raft_consensus.cc:2272] T c8983e4294f94da38f38ac2f6ea001fe P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.824394  8021 tablet_replica.cc:333] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.824919  8021 raft_consensus.cc:2243] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.825677  8021 raft_consensus.cc:2272] T de4868f214f94bbfa2b0a721fd77d0f5 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.827611  8021 tablet_replica.cc:333] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.828162  8021 raft_consensus.cc:2243] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.828588  8021 raft_consensus.cc:2272] T c4400e1db38f4227bc7ddcde78d93a11 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.830435  8021 tablet_replica.cc:333] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.830955  8021 raft_consensus.cc:2243] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.831696  8021 raft_consensus.cc:2272] T 68989224c86646159ba1d1a5e722f160 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.833644  8021 tablet_replica.cc:333] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.834230  8021 raft_consensus.cc:2243] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.835143  8021 raft_consensus.cc:2272] T 031f8922905b4356aab8310161abf4f6 P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.838701  8021 tablet_replica.cc:333] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.839321  8021 raft_consensus.cc:2243] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.839808  8021 raft_consensus.cc:2272] T ed00071d9a064800985eeae64cdda3e7 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.842204  8021 tablet_replica.cc:333] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.842777  8021 raft_consensus.cc:2243] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.843261  8021 raft_consensus.cc:2272] T a20d75390629460099ed0728baefb3ab P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.845347  8021 tablet_replica.cc:333] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.845927  8021 raft_consensus.cc:2243] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.846349  8021 raft_consensus.cc:2272] T 1f35f40f03074dd4862bc0e80fd8caed P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.848335  8021 tablet_replica.cc:333] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.848869  8021 raft_consensus.cc:2243] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.849679  8021 raft_consensus.cc:2272] T c4c055085a1d4959a878054bb8fd614c P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.851686  8021 tablet_replica.cc:333] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.852293  8021 raft_consensus.cc:2243] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.852720  8021 raft_consensus.cc:2272] T 57c432b444fb4596a3a38210d8b8936d P 47483c02b1e348278a40781fdc137f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.854789  8021 tablet_replica.cc:333] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.855317  8021 raft_consensus.cc:2243] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.855815  8021 raft_consensus.cc:2272] T e37171da97ae48e3a77e03d033b6d23c P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.857743  8021 tablet_replica.cc:333] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.858265  8021 raft_consensus.cc:2243] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.858704  8021 raft_consensus.cc:2272] T 39c48d1152f54851b1209458b609506e P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.860740  8021 tablet_replica.cc:333] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.861261  8021 raft_consensus.cc:2243] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.862018  8021 raft_consensus.cc:2272] T 0864698179204de6a0c3ab6f41959297 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.864145  8021 tablet_replica.cc:333] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.864619  8021 raft_consensus.cc:2243] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.865412  8021 raft_consensus.cc:2272] T 3016133635d74bcdbf0b50a9086778ab P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.867326  8021 tablet_replica.cc:333] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96: stopping tablet replica
I20260709 13:08:03.867794  8021 raft_consensus.cc:2243] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.868227  8021 raft_consensus.cc:2272] T 25af7d7302394721a33cd7017a84bea3 P 47483c02b1e348278a40781fdc137f96 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.912324  8021 tablet_server.cc:196] TabletServer@127.7.213.66:0 shutdown complete.
I20260709 13:08:03.945781  8021 tablet_server.cc:179] TabletServer@127.7.213.67:0 shutting down...
I20260709 13:08:03.974215  8021 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:03.974926  8021 tablet_replica.cc:333] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:03.975620  8021 raft_consensus.cc:2243] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.976152  8021 raft_consensus.cc:2272] T 406df04366c743bc8a4b4542161f14f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.978183  8021 tablet_replica.cc:333] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:03.978758  8021 raft_consensus.cc:2243] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.979190  8021 raft_consensus.cc:2272] T 7490273fa34949f08f5b3aa510be570b P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.981230  8021 tablet_replica.cc:333] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:03.981763  8021 raft_consensus.cc:2243] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.982522  8021 raft_consensus.cc:2272] T 90a9ec28276348f787cd28248f7c55ef P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.984540  8021 tablet_replica.cc:333] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:03.985113  8021 raft_consensus.cc:2243] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.985536  8021 raft_consensus.cc:2272] T c8983e4294f94da38f38ac2f6ea001fe P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.987455  8021 tablet_replica.cc:333] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:03.988065  8021 raft_consensus.cc:2243] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.988467  8021 raft_consensus.cc:2272] T 57c432b444fb4596a3a38210d8b8936d P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.990332  8021 tablet_replica.cc:333] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:03.990875  8021 raft_consensus.cc:2243] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.991321  8021 raft_consensus.cc:2272] T e37171da97ae48e3a77e03d033b6d23c P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.993201  8021 tablet_replica.cc:333] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:03.993700  8021 raft_consensus.cc:2243] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:03.994560  8021 raft_consensus.cc:2272] T 39c48d1152f54851b1209458b609506e P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.996696  8021 tablet_replica.cc:333] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:03.997232  8021 raft_consensus.cc:2243] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:03.997697  8021 raft_consensus.cc:2272] T 0864698179204de6a0c3ab6f41959297 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:03.999593  8021 tablet_replica.cc:333] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.000141  8021 raft_consensus.cc:2243] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:04.000576  8021 raft_consensus.cc:2272] T 1f35f40f03074dd4862bc0e80fd8caed P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:04.002559  8021 tablet_replica.cc:333] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.003095  8021 raft_consensus.cc:2243] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:04.003523  8021 raft_consensus.cc:2272] T a20d75390629460099ed0728baefb3ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:04.005456  8021 tablet_replica.cc:333] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.005947  8021 raft_consensus.cc:2243] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:04.006366  8021 raft_consensus.cc:2272] T 68989224c86646159ba1d1a5e722f160 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:04.008273  8021 tablet_replica.cc:333] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.008936  8021 raft_consensus.cc:2243] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:04.009383  8021 raft_consensus.cc:2272] T 031f8922905b4356aab8310161abf4f6 P 3f652e73eb8045f6bcdcfe70e5047960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:04.011097  8021 tablet_replica.cc:333] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.011590  8021 raft_consensus.cc:2243] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:04.012322  8021 raft_consensus.cc:2272] T ed00071d9a064800985eeae64cdda3e7 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:04.014243  8021 tablet_replica.cc:333] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.014735  8021 raft_consensus.cc:2243] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:04.015146  8021 raft_consensus.cc:2272] T 3016133635d74bcdbf0b50a9086778ab P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:04.017155  8021 tablet_replica.cc:333] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.017735  8021 raft_consensus.cc:2243] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:04.018234  8021 raft_consensus.cc:2272] T 25af7d7302394721a33cd7017a84bea3 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:04.020278  8021 tablet_replica.cc:333] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.020805  8021 raft_consensus.cc:2243] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:04.021299  8021 raft_consensus.cc:2272] T c4c055085a1d4959a878054bb8fd614c P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:04.023265  8021 tablet_replica.cc:333] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.023804  8021 raft_consensus.cc:2243] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:04.024253  8021 raft_consensus.cc:2272] T de4868f214f94bbfa2b0a721fd77d0f5 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:04.026067  8021 tablet_replica.cc:333] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960: stopping tablet replica
I20260709 13:08:04.026510  8021 raft_consensus.cc:2243] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:04.027238  8021 raft_consensus.cc:2272] T c4400e1db38f4227bc7ddcde78d93a11 P 3f652e73eb8045f6bcdcfe70e5047960 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:04.590278 19198 debug-util.cc:398] Leaking SignalData structure 0x7b080012bd60 after lost signal to thread 8024
W20260709 13:08:04.591274 19198 debug-util.cc:398] Leaking SignalData structure 0x7b080003a020 after lost signal to thread 18960
W20260709 13:08:05.066329  8021 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 19198)
W20260709 13:08:06.066865  8021 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 19198)
W20260709 13:08:07.067287  8021 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 19198)
I20260709 13:08:07.693974  8021 tablet_server.cc:196] TabletServer@127.7.213.67:0 shutdown complete.
I20260709 13:08:07.732206  8021 master.cc:562] Master@127.7.213.126:33533 shutting down...
I20260709 13:08:07.762424  8021 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:07.762988  8021 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:07.763283  8021 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2acd56cd48af490797ad208fbb71b121: stopping tablet replica
I20260709 13:08:07.784729  8021 master.cc:584] Master@127.7.213.126:33533 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (17780 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (32344 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (329460 ms total)
[  PASSED  ] 8 tests.
I20260709 13:08:07.843286  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 412 messages since previous log ~20 seconds ago
I20260709 13:08:07.843511  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 412 messages since previous log ~20 seconds ago
I20260709 13:08:07.843676  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 407 messages since previous log ~20 seconds ago
I20260709 13:08:07.843941  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~41 seconds ago
I20260709 13:08:07.844107  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~41 seconds ago
I20260709 13:08:07.844252  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:5227: suppressed but not reported on 1 messages since previous log ~126 seconds ago
I20260709 13:08:07.844415  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 9 messages since previous log ~156 seconds ago
I20260709 13:08:07.844564  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 2 messages since previous log ~130 seconds ago
I20260709 13:08:07.844774  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 7 messages since previous log ~4 seconds ago
I20260709 13:08:07.845041  8021 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~16 seconds ago
